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Introduction 


The  intent  of  this  research  was  to  examine  the  utility  and  merit  of  a  novel  system  of 
intuitive  symbols  (a  mnemonie  strategy)  in  conjunction  with  the  training  of  emergency 
procedures  to  U.S.  Army  aviation  students  in  order  to  faeilitate  the  accurate  reeall  of  those 
proeedures.  Ever  sinee  earth-bound  man  deeided  to  take  to  the  skies  in  flying  maehines,  he  has 
had  to  deal  with  emergency  situations  created  by  the  oeeasional  malfunction  or  failure  of  aireraft 
components.  The  steps  taken  to  correct,  or  at  least  ameliorate,  the  eondition  must,  by  neeessity, 
be  performed  in  an  established  manner  and/or  sequenee.  To  this  end,  all  student  pilots  must 
learn  (memorize)  and  praetiee  emergency  procedures  (ordered  steps)  established  for  speeifie 
emergeney  events  for  the  partieular  model  of  aircraft  in  which  they  are  being  trained.  As  an 
example,  eurrently,  pilots  of  the  Army’s  UH-60  Blaek  Hawk  helieopter  are  required  to  learn 
proeedural  steps  for  up  to  56  emergency  situations  (Headquarters,  Department  of  the  Army, 
2003). 


Baekground 

The  first  step  in  the  U.S.  Army’s  eurrent  praetiee  of  teaehing  aviation  emergeney 
proeedures  is  to  require  student  pilots  to  learn  the  textual  proeedures  through  rote  memorization. 
No  standardized  or  formal  guidance  or  teehniques  are  provided  as  best  praetiees  in  whieh  to 
learn  them.  Therefore,  conventional  training  eonsists  of  the  requirement  for  student  pilots  to 
learn  these  steps  from  textual  lists  provided  in  the  model-speeifie  operator’s  manuals.  The  U.S. 
Army  is  partieularly  demanding  of  their  student  and  graduate  pilots  in  that  it  requires  the 
memorization  of  an  extensive  number  of  emergency  procedures  (many  more  than  the  other 
military  serviees  require),  eaeh  of  whieh  usually  eontains  multiple,  purposefully-ordered  steps. 
The  satisfaetory  demonstration  of  emergency  procedures  reeall  and  performanee  is  a  requirement 
for  the  completion  of  flight  sehool  and  the  annual  evaluations  of  graduate  pilots  (Headquarters, 
Department  of  the  Army,  1996). 

The  reeall  of  the  multitude  of  emergeney  proeedures  required  is  a  daunting  effort.  Aetual 
aireraft  emergeneies  are  infrequent  and  some  are  quite  rare.  Henee,  without  active  reeurring 
maintenanee  (praetiee)  of  the  eonventionally-leamed  textual  proeedures,  the  ability  of  reeall 
begins  to  deeay  signifieantly  over  a  relatively  short  period  of  time  whieh  results  in  the  inability 
to  respond  aeeurately  and  instantly  in  the  event  of  an  aetual  emergeney,  the  eonsequenees  of 
whieh  ean  be  eatastrophie.  This  premise  is  supported  by  the  results  of  an  anonymous  survey 
(Estrada  and  Dumond,  2006)  of  Army  aviators  (n  =  194)  eondueted  by  the  U.S.  Army 
Aeromedieal  Researeh  Eaboratory  (USAARL)  that  indieate  that  16%  of  the  respondents  report 
reeiting  or  praetieing  their  emergeney  proeedures  daily,  while  48%  eonvey  that  they  praetiee  at 
least  weekly.  This  reported  need  to  praetiee  so  frequently  even  after  many  years  of  such  practice 
is  indieative  of  a  memory  strategy  that  appears  to  be  ineffeetive  at  transforming  these  proeedural 
steps  to  reliably  retrievable  memory.  Interestingly,  8%  of  the  respondents  are  not  satisfied  with 
their  eurrent  memorization  methods  while  66%  indieate  an  interest  in  learning  new  memorization 
strategies  to  help  retain  their  emergeney  proeedures. 

The  Instruetor  Pilot  Handbook  (U.S.  Army  Aviation  Center,  2000)  provides  three 
theories  that  aeeount  for  the  forgetting  [of  emergeney  proeedures];  disuse,  interference,  and 
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repression.  Disuse  refers  to  information  that  is  not  often  used.  This  may  explain  the  diffieulty  in 
remembering  some  emergeney  proeedures  that  are  the  least  frequently  praetieed.  Interference 
deseribes  what  may  happen  when  similar  information  interferes  with  the  memory  of  information 
previously  learned.  New  events  or  experienees  ean  displaee  previous  ones.  Many  “seasoned” 
aviators  experienee  interferenee  when  the  steps  of  emergeney  proeedures  are  ehanged  over  time. 
Repression  is  said  to  oeeur  when  unpleasant  or  anxiety-produeing  material  is  unintentionally 
suppressed  by  an  individual.  Perhaps  not  eommon,  but  repression  of  information  is  eertainly 
possible  when  learning  is  eomplieated  or  diffieult,  as  is  eertainly  possible  and  probable  in  flight 
training. 

There  are  a  number  of  theories  suggesting  the  meehanisms  in  whieh  the  brain  transforms 
a  pereeption  (objeets,  text)  to  meaningful  pereeption  (eoneepts)  to  a  retrievable  memory.  These 
will  be  explored  in  further  detail  later.  Regardless  of  the  theoretieal  meehanisms,  the  purpose  of 
any  learning  is  to  transfer  new  information  from  working  or  short-term  memory  (a  temporary 
store)  to  long-term  memory,  memory  that  has  no  eapaeity  limits  and  holds  information  from 
minutes  to  an  entire  lifetime  (Reed,  2004).  Previous  researeh  and  experienee  has  demonstrated 
that  providing  students  with  instruetions  on  the  use  of  memorization  teehniques  (mnemonie 
strategies)  has  resulted  in  improvements  in  their  ability  to  reeall  learned  information  (Carney  and 
Levin,  2003;  Kleinheksel  and  Summy,  2003;  Hwang  and  Levin,  2002;  Carney  and  Levin,  2002; 
Cox,  2001).  Mnemonie  strategies  are  systematie  proeedures  for  enhaneing  memory  (Mastropieri 
and  Seruggs,  1998)  and  are  used  to  faeilitate  the  aequisition  of  faetual  information  beeause  they 
assist  in  the  memory  eneoding  proeess,  either  by  providing  familiar  connections  or  by  creating 
new  connections  between  to-be-remembered  information  and  the  learner’s  prior  knowledge 
(Levin  and  Levin,  1990). 


An  intuitive  pictorial  system 

With  the  above  said,  this  research  sought  to  examine  the  utility  and  merit  of  a  novel 
system  of  intuitive  pictures  (the  Intuitive  Pictorial  System  or  IPS)  taught  as  a  mnemonic  strategy 
to  facilitate  learning  and  accurate  recall  of  complex  emergency  procedures.  In  addition,  analyses 
were  conducted  to  determine  the  effectiveness  of  this  novel  mnemonic  strategy  with  respect  to 
the  aviators’  levels  of  flight  experience,  their  learning  style  preferences,  and  their  overall 
subjective  assessments  of  the  system. 

The  pictures  that  comprise  the  IPS  are  characterized  as  intuitive  as  they  are  formed  with 
symbols  representing  aircraft  parts  and  systems  and  are  presumed  easily  and  immediately 
recognizable  to  pilots.  Thus,  they  require  no  cognitive  effort  in  determining  their  meanings.  For 
example,  the  capital  T  is  easily  visualized  as  a  helicopter  rotor  system  (the  top  of  the  T  represents 
the  rotor  blades).  Any  picture  with  a  T  would  necessarily,  and  intuitively,  involve  the  rotor 
system.  The  intuitive  pictorial  system  (its  symbols  and  their  meanings)  is  available  in  Appendix 
A. 


The  IPS  is  novel  in  that  1)  it  is  the  creation  of  the  researcher,  and  2)  to  this  day,  there  has 
never  been  an  attempt  to  introduce  any  structured,  formal  mnemonic  strategy  system  to  the 
learning  of  Army  aviation  emergency  procedures.  This  novel  approach  to  teaching  aviation 
emergency  procedures  may  be  an  important  innovation  to  current  Army  teaching  techniques  and 
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a  notable  contribution  to  aviation  training  and  safety  with  possible  applications  to  the  other 
military  services  and  the  civil  aviation  community. 


Research  problem 

A  commonly-expressed  problem  by  U.S.  Army  pilots  is  that  without  recurrent  practice 
and  recitation,  they  tend  to  lose  their  ability  to  retain  and  recall  important  aircraft  emergency 
procedures  over  a  relatively  short  period  of  time.  Evidence  of  this  problem  is  provided  by  the 
results  of  a  USAARL  emergency  procedures  survey  (Estrada  and  Dumond,  2006)  from  which  it 
was  discovered  that  27.8%  of  the  respondents  disagreed  that  learning  new  emergency  procedures 
was  easy.  In  addition,  68%  reported  that  they  must  study  or  practice  reciting  their  procedures 
often,  with  77.9%  conveying  that  they  must  practice  more  often  than  once  every  two  weeks  in 
order  to  maintain  proficiency.  No  known  efforts  have  been  made  by  those  responsible  for  Army 
aviation  training  to  explore  and  make  use  of  innovative  training  or  mnemonic  techniques  in  order 
to  improve  the  current  state. 

U.S.  Army  pilots  are  required  to  learn  a  multitude  of  emergency  procedures  containing 
multiple  textual  steps  and  are  then  required  to  recall  them  verbatim  in  preparation  for  simulated 
or  actual  emergency  situations.  The  syntax  of  the  steps  tends  to  be  structurally  cumbersome  and 
may  contribute  to  the  difficulty  in  memorizing  them.  The  maintenance  of  these  textual 
procedures,  even  after  they  are  “learned,”  requires  hours  of  study  “refreshing  one’s  memory”  in 
order  to  maintain  proficiency,  and  yet  experience  shows  that  such  efforts  do  not  assure  accuracy 
and  timeliness  of  recall.  In  a  reasonable  comparison,  a  study  of  the  memory  strategies  of 
professional  actors  by  Noice  (1992)  discovered  that  there  was  unanimous  agreement  among  the 
actors  that  they  do  not  memorize  their  lines  in  a  rote -type  fashion  (as  the  Army  requires).  Even 
if  the  Army  student  pilot  progresses  beyond  rote  memorization  (the  lowest  level  of  learning)  to 
the  higher  levels  of  understanding,  application,  and  correlation  (U.S.  Army  Aviation  Center, 
2000),  the  form  (the  textual  format)  in  which  the  emergency  procedures  are  presented,  combined 
with  the  disuse  or  infrequency  in  performing  the  procedures  once  the  student  graduates  from 
flight  school,  contributes  to  the  general  tendency  to  forget  the  specific  procedures.  It  is  common 
knowledge  among  graduate  pilots  that  the  memory  of  these  steps  is  quite  perishable  and  requires 
a  considerable  amount  of  time  rehearsing  even  after  years  of  experience  and  practice.  The 
consequences  of  forgetting  them  can  range  from  an  unsatisfactory  grade  during  flight  evaluations 
to  the  more  serious  inability  to  respond  to  and  perform  correctly  during  an  actual  emergency 
situation  (Cumbie,  2003). 

According  to  Bellezza  (1992),  memory  experts,  called  mnemonists,  learn  to  create  mental 
pictures  that  endure  in  the  mental  space.  Thus,  the  IPS  depicts  each  emergency  situation  and  its 
procedural  steps  in  a  single  pictorial  form.  Bellezza  writes  that  it  appears  that  experiencing  and 
manipulating  vivid  visual  images  is  the  manner  by  which  mnemonists  memorize  information.  In 
an  article  regarding  visual  puns  as  illustrations.  Abed  (1994)  notes,  “Initially,  capturing  the 
attention  of  the  learner  is  a  crucial  step  that  interactive  illustrations  are  capable  of  achieving.” 

The  expectation  was  that  the  IPS  was  capable  of  serving  that  purpose  and  would  be  helpful  in 
improving  a  pilot’s  ability  to  retain  emergency  procedures  over  time. 
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Research  questions 


The  specific  research  questions  that  directed  this  study  were: 

1 .  Will  the  employment  of  the  IPS  demonstrate  improved  retention  and  recall  of 
emergency  procedures  over  that  of  the  traditional  teaching  method  in  the  overall 
sample  population? 

2.  Is  there  a  difference  in  the  recall  performance  between  those  taught  using  the 
traditional  method  and  those  taught  using  the  IPS  method  with  respect  to  experience 
levels  (highly-experienced  instructor  pilots,  minimally-experienced  student  pilots, 
and  naive  students)? 

3.  Is  there  a  difference  in  the  recall  performance  between  those  taught  using  the 
traditional  method  and  those  taught  using  the  IPS  method  with  respect  to  individual 
learning  style  preferences  (visual,  aural,  read/write,  and  kinesthetic)? 

4.  Does  recall  performance  have  any  correlation  to  the  manner  in  which  the  IPS  is 
assessed? 

5.  Do  individual  learning  style  preferences  have  an  effect  on  the  aviators’  subjective 
assessment  of  the  IPS  (pictorial  mnemonics)  as  an  aid  in  memorizing  textual 
emergency  procedures? 

6.  Do  experience  levels  have  an  effect  on  the  aviators’  subjective  assessment  of  the  IPS 
(pictorial  mnemonics)  as  an  aid  in  memorizing  textual  emergency  procedures? 

7.  Is  the  IPS  assessed  positively  by  the  majority  of  those  in  the  treatment  groups  as  a 
useful  pictorial  mnemonic  for  remembering  aviation  emergency  procedures? 

8.  After  instruction,  are  the  symbols  that  make  up  the  IPS  easily  recognized  (intuitive) 
and  thus,  remembered  (have  memorable  qualities)? 

9.  Is  there  an  association  between  committed  study  time  from  Day  I  to  Day  7  and 
posttest  performances? 

Literature  review  and  theoretical  background 

Too  many  times  educators  tell  students  what  to  learn  yet  fail  to  teach  students  how  to 
learn.  Ashman  and  Conway  (1997)  describe  a  paradigm  shift  in  learning  since  the  mid-1980s 
from  one  in  which  the  “emphasis  is  on  content  and  acquiring  a  body  of  right  knowledge”  to  one 
in  which  the  “emphasis  is  on  learning  how  to  learn.”  Knowing  “how  to  learn”  involves  the 
learning  of  strategies.  Strategies  refer  to  the  many  methods  in  which  we  take  in  (encode),  store, 
and  retrieve  (decode)  information.  Unfortunately,  strategies  used  for  enhancing  learning  are  not 
an  innate  student  ability.  “Teachers  must  build  into  the  learning  context  retrieval  cues  that  will 
likely  be  present  when  the  students  need  to  recall  the  concept”  (Squire  and  Kandel,  quoted  by 
King-Friedrichs,  2001). 

Moley,  et  al.  (quoted  by  Cox,  2001)  reported  that  strategies  are  seldom  taught  in  the 
classroom.  Results  of  a  study  by  Cox  indicated  that  many  young  elementary  school  children, 
without  training,  do  not  use  an  organized  sorting  strategy  to  aid  memorization.  In  his 
experiment,  Cox  found  that  untrained  children  used  sorting  strategies  that  were  “haphazard  at 
best,  passive  at  worst,  and  they  produced  sorts  that  were  reliably  judged  as  random. . .” 
Predictably,  Moley  and  Hart  (1992)  write  that  students  of  those  teachers  who  often  suggested 
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strategies  showed  better  maintenance  and  more  deliberate  use  of  the  trained  strategies  than  did 
children  whose  teachers  rarely  made  strategy  suggestions. 

Studies  into  the  effectiveness  of  mnemonic  strategies  indicate  that  considerable  success  is 
achieved  in  the  retention  and  recall  of  the  information.  How  much  is  remembered  depends  on 
what  was  already  known  about  what  is  being  remembered  (Kuhn  2000).  Mastropieri  and 
Scruggs  (1998)  write  that  the  particular  task  in  developing  mnemonic  strategies  is  to  find  a  way 
to  relate  new  information  to  information  students  already  have  locked  in  long-term  memory. 

They  continue,  “If  we  can  make  a  firm  enough  connection,  the  memory  will  last  a  very  long 
time.”  In  Cox’s  (2001)  study,  his  findings  confirmed  the  many  common  findings  of  earlier 
work:  that  mnemonic  strategies  are  not  difficult  to  train  to  students  and  that  once  learned, 
produce  measurable  improvements  in  their  recall  performance  over  time.  Cox  observes  that 
when  left  to  their  own  devices,  children  will  construct  their  own  haphazard  strategies.  These 
variations,  he  notes,  in  the  effectiveness  of  these  self-constructed  strategies  undoubtedly 
contribute  to  the  wide  individual  differences  in  their  development.  In  several  experiments.  Levin 
and  his  associates  (Carney  and  Levin,  2003;  Hwang  and  Levin,  2002;  Carney  and  Levin,  2002) 
have  demonstrated  the  utility  of  employing  pictorially-based  mnemonic  strategies.  In  each 
experiment,  results  have  indicated  the  statistically  significant  superior  performance  of  those 
employing  a  taught  strategy  over  that  of  an  untrained  control  group. 

Theoretical  framework 

The  suggestion  of  employing  a  novel  training  technique  or  teaching  approach  over  that  of 
a  traditional  method  requires  evidence  that  the  novel  system  will  result  in  improved  knowledge 
retention  or  recall  performance.  In  this  study’s  specific  case,  the  IPS  must  demonstrate  an 
improvement  in  the  users’  memory  for  the  information.  The  novelty  of  the  approach  in 
facilitating  aviation  emergency  procedure  learning  and  recall  through  an  intuitive  pictorial 
system  unavoidably  begins  with  a  change  from  the  traditional  method  in  the  presentation  of  the 
information,  specifically,  from  a  textual  format  to  that  of  a  pictorial  representation  of  the 
information.  This  alteration  in  form  presumably  influences  visual  perception,  which  in  turn 
potentially  affects  the  visual  and  conceptual  cognition  of  the  information  (the  learning).  The 
affected  cognitive  process  may  change  the  manner  and  speed  in  which  the  knowledge  is  retained 
in  the  memory  and  may  ultimately  have  an  effect  on  the  ability  to  recall  the  information  when 
needed  or  desired.  In  other  words,  it  is  impossible  to  study  “any  type  of  memory  processing  in 
isolation  from  the  other  aspects  of  the  developing  cognitive  system”  (Schneider  and  Bjorklund, 
1998,  p.  492).  Kuhn  (2000)  puts  it  this  way: 

Although  memory  has  long  been  regarded  as  a  central  and  well-defined  topic 
within  the  field  of  cognitive  development,  developments  in  and  related  to  the 
study  of  memory  increasingly  suggest  that  the  study  of  memory  needs  to  be 
situated  in  a  number  of  broader  conceptual  and  research  contexts. 

That  being  said,  the  following  is  an  examination  and  review  of  the  theoretical  bases  for 
suggesting  that  a  pictorially-based,  mnemonic  strategy  might  be  effective  and  covers  a  wide 
range  of  interrelated  domains. 
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Learning 


Learning  ean  be  deseribed  as  the  gaining  of  knowledge,  understanding,  or  skill 
(Webster’s  Ninth  New  Collegiate  Dictionary,  1985)  with  an  outcome  of  a  change  in  behavior 
[overt  or  subtle]  as  a  result  of  the  experience  (U.S.  Army  Aviation  Center,  2000).  It  is  said  to 
occur  when  the  individual  intentionally  pays  attention  to  the  contents  of  working  memory 
leading  it  to  be  absorbed  into  long-term  memory  (Herrmann,  Raybeck,  and  Gruneberg,  2002). 
(More  on  memory  stores  later.) 

What  is  learned  depends  very  much  on  how  it  is  learned.  According  to  the  Instructor 
Pilot’s  Handbook  (U.S.  Army  Aviation  Center,  2000),  learning  occurs  through  the  perceptions  of 
the  five  senses:  sight,  hearing,  touch,  smell,  and  taste.  These  perceptions  result  when  a  person 
gives  meaning  to  the  sensations  and  are  the  basis  for  all  learning.  It  is  important  to  note  that  a 
person’s  perceptions  are  influenced  by  several  factors  (or  trainee  characteristics.  May  and 
Kahnweiler,  2000)  which  do  affect  the  gaining  of  knowledge,  understanding,  or  skill.  These 
factors  are  a  person’s  1)  physical  organism  (the  condition  of  the  senses),  2)  basic  needs  (physical 
and  psychological),  3)  goals  and  values  (motivation  and  personal  experiences),  4)  self-concept 
(positive/negative),  5)  time  and  opportunity  (to  perceive),  and  6)  element  of  threat  (fear 
adversely  affects  perception)  (U.S.  Army  Aviation  Center).  Considering  the  influence  of  these 
factors  on  perceptions  and  therefore,  learning.  May  and  Kahnweiler  submit  that  learning 
retention  is  also  impacted  by  the  training  design,  trainee  characteristics,  and  learning 
environment  variables.  In  order  to  be  retained,  the  information,  once  perceived,  must  be 
processed  and  stored  through  a  cognitive  process. 

Cognitive  theory 

S.K.  Reed  (2004)  defines  cognition  simply  as  the  acquisition  of  knowledge.  Such 
acquisition,  he  writes,  involves  many  mental  skills.  Ashman  and  Conway  (1997,  p.  41) 
elaborate,  “It  involves  taking  in,  storing,  retrieving,  transforming,  and  manipulating  information. 
The  process  necessarily  entails  perception,  awareness,  judgment,  the  understanding  of  emotions, 
memory,  and  learning.”  (Note  that  for  the  discourse  within  this  section,  the  author  drew  heavily 
from  books  by  Ashman  and  Conway,  1997,  and  Reed,  2004.) 

The  study  of  thinking  and  learning  has  been  influenced  by  two  major  traditions: 
behaviorism  and  human  memory  research.  The  foundation  of  behaviorism  is  that  behavior  is 
influenced  by  the  environment  in  which  the  learning  occurs.  It  has  its  base  in  the  work  of  E.L. 
Thorndike,  I.P.  Pavlov,  B.F.  Skinner,  and  J.B.  Watson.  All  have  had  a  profound  effect  and 
impact  on  today’s  educational  process.  Thorndike  and  Pavlov  manipulated  the  environmental 
conditions  of  animals.  Thorndike’s  “law  of  effecf  ’  (using  cats)  was  based  on  the  premise  that  a 
correct  response  would  lead  to  reinforcement  and  learning  would  occur.  Pavlov,  in  his  famous 
dog  experiments,  demonstrated  that  responses  to  conditioned  stimuli  can  be  learned  and  become 
extinct  if  not  maintained.  Skinner’s  methods,  known  as  operant  conditioning,  were  tried  first  on 
animals  and  later  on  humans.  Operant  conditioning  is  characterized  by  the  process  of  learning 
from  one  step  to  another,  with  each  new  step  dependent  on  a  previously  learned  step.  For 
Skinner,  learning  was  the  result  of  the  reinforcement  or  punishment  within  a  context  controlled 
by  the  teacher.  J.B.  Watson,  originator  of  the  term  “behaviorism,”  argued  that  researchers 
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should  study  only  what  could  be  directly  observed.  Basieally,  Watson  promoted  the  study  of  the 
stimulus-response  approaeh  (how  people  responded  to  stimuli)  without  regard  for  the  thought 
proeess  involved  in  the  response.  The  problem  with  sueh  an  approaeh  is  that  it  does  not  reveal 
what  the  person  does  with  the  information  presented  in  the  stimulus. 

A  ehange  from  the  stimulus-response  to  the  information-proeessing  approaeh  began 
during  the  1950s.  The  information-proeessing  approaeh  (influeneed  by  the  eomputer  metaphor) 
is  eharacterized  by  separate  stages  (aequisition,  storage,  retrieval,  and  use)  and  attempts  to  depiet 
what  happens  as  the  information  is  proeessed  through  these  stages  (Haber,  1969).  In  other 
words,  this  methodology  attempts  to  reveal  what  the  person  does  with  the  information  presented 
in  a  stimulus.  The  information-proeessing  approaeh  has  gathered  momentum  sinee  the  1950s  and 
has  gained  aeceptanee  by  today’s  eognitive  psyehologists  and  researehers.  It  has  become  the 
foundation  from  whieh  theoretieal  frameworks  (models)  of  cognitive  architeeture  have  been 
developed. 

Information  proeessing:  the  cognitive  arehiteeture 

Theoretieal  frameworks  for  the  brain’s  eognitive  arehiteeture  have  been  presented  and 
debated  for  many  deeades.  Future  researeh  may  diseover  the  aetual  meehanisms  of  eognitive 
proeessing,  but  at  present,  allusions  to  theoretieal  eoneepts  are  the  only  reeourse.  An  in-depth 
examination  and  assessment  of  every  cognitive  theory  is  not  useful  to  this  research;  however,  a 
eoneeptual  framework  is  eertainly  helpful  in  illustrating  the  speeulated  proeess.  A  diseussion  of 
the  most  prevailing  theories  of  the  reeent  past  follows. 

In  1972,  renowned  researcher  of  human  memory,  F.  1.  M.  Craik,  and  eo-author,  R.  S. 
Loekhart,  published  a  review  of  the  prevailing  “multistore”  theories  of  memory  whieh  serves  as 
the  basis  for  some  of  today’s  eoneepts.  The  general  multistore  model  presumed  that  pereeived 
information  was  held  in  memory  stores  at  various  points  in  the  cognitive  system.  After 
pereeption  (the  sensory  stores),  the  information  was  plaeed  in  a  short-term  store  (STS)  and  then 
eould  be  transferred  into  and  retained  in  a  more  permanent  long-term  store  (LTS).  Researeh 
demonstrated  the  distinctions  between  STS  and  LTS.  It  was  aceepted  that  STS  had  a  limited 
eapaeity,  whereas  LTS  had  no  known  limit.  Researeh  into  the  STS’s  limited  eapaeity  was  not 
elear  as  to  the  limiting  faetor:  storage  eapaeity  or  the  rate  at  whieh  the  proeessor  eould  perform 
eertain  operations.  Craik  and  Loekhart  favored  the  latter:  that  the  limiter  was  the  proeessing  rate 
and  not  storage  eapaeity.  They  wrote  that  “the  eoneept  of  eapaeity  is  to  be  understood  in  terms 
of  a  limitation  on  proeessing;  limitations  of  storage  are  held  to  be  a  direet  consequenee  of  this 
more  fundamental  limitation.” 

Finally,  regarding  memory  and  reeall,  Craik  and  Loekhart  (1972)  noted  that  many 
theorists  of  the  period  agreed  that  analysis  of  information  proeeeded  through  a  series  of  sensory 
stages  to  levels  assoeiated  with  matching  or  pattern  reeognition  and  finally  to  semantie- 
assoeiative  stages  of  stimulus  enriehment  [eneoding].  A  result  of  this  pereeptual  analysis  was 
the  creation  of  a  memory  traee.  The  persistenee  of  the  traee  (its  ability  to  be  used  and/or 
aeeessed  later)  was  a  matter  of  debate.  Craik  and  Loekhart  suggested  that  the  traee  persistenee 
was  a  function  of  the  depth  of  analysis,  with  deeper  levels  of  analysis  assoeiated  with  more 
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elaborate,  longer  lasting  and  stronger  traces,  and  that  retention  was  a  function  of  this  depth  of 
process. 

Another  plausible  cognitive  theory  that  persists  and  continues  to  be  cited  in  the  current 
literature  (May  and  Kahnweiler,  2000;  Pirolli  and  Card,  1999;  Anderson  and  Reder,  1999a, 
1999b;  Blessing  and  Anderson,  1996)  is  J.  R.  Anderson’s  ACT  or  Adaptive  Control  of  Thought, 
first  described  in  1983.  The  theory  has  evolved  over  time  and  has  incorporated  and  expounded 
on  a  number  of  assumptions  and  phenomena  since  its  original  form.  A  simplified  explanation  of 
the  theory  describes  a  system  wherein  there  are  three  major  structural  components  and  their 
interlinking  processes:  working  memory,  declarative  memory,  and  production  memory. 

Referring  to  the  ACT  theory.  May  and  Kahnweiler  write  that  skill  acquisition  is  believed  to 
proceed  through  three  stages:  a  cognitive  stage,  in  which  a  description  of  the  procedure  is  learned 
(declarative  knowledge);  an  associative  stage,  in  which  facts  are  compiled  and  integrated  with  a 
method  for  performing  the  skill  (procedural  knowledge);  and  an  autonomous  stage,  in  which  the 
skill  becomes  more  fluent  and  automatic.  Basically,  information  from  the  outside  world  is 
encoded  into  a  working  memory  which  is  currently  accessible,  followed  by  a  storage  process  that 
creates  permanent  records  in  the  long-term  declarative  memory  (Anderson,  1983).  Previously 
stored  information  in  the  declarative  memory  is  accessed  by  the  working  memory  when  retrieval 
of  information  is  necessary  or  desired.  The  production  memory  is  engaged  whenever 
performance  and  execution  processes  are  necessary.  Associated  or  related  permanent 
information  is  connected  by  traces.  According  to  the  theory,  a  trace  once  formed  is  not  lost,  but 
its  strength  may  decay  [due  to  infrequent  use  or  access]. 

Strengthening  and  retrieving  traces 

As  indicated,  the  longer  the  time  a  trace  is  left  unused,  the  harder  it  is  to  find  amidst 
everything  else.  According  to  Herrmann,  Raybeck,  and  Gruneberg  (2002),  the  strength  of 
memory  traces  can  be  increased  through  the  mental  manipulations  that  order  and  organize 
thoughts  to  assist  registration,  retention,  and  remembering.  These  trace-strengthening 
manipulations  work  1)  by  intensifying  learner  attention  during  the  learning,  retention,  and 
retrieval  phases  of  the  memory  process  and  2)  by  involving  the  rehearsal  of  the  items  to  be 
remembered. 

Strategies  for  enhancing  the  retrieval  of  information  include  techniques  in  association 
(associating  a  known  concept/word  with  a  new  one),  clustering  (grouping  related  information), 
imagery,  location,  mnemonic  devices,  and  visualization  (Goll,  2004;  Mindtools,  n.d.). 

Herrmann,  Raybeck,  and  Gruneberg  (2002),  inform  1)  that  association  manipulations  relate 
different  traces  with  each  other,  including  associating  verbally,  associating  present  information 
with  past  information,  and  associating  meaningful,  phonetic,  and/or  visual  relationships;  and  2) 
that  traces  are  easier  to  retrieve  when  surprising  or  interesting  attributes  are  assigned  to  the 
information,  i.e.,  size,  shape,  color,  judgments,  description. 

In  his  2004  book.  Cognition:  Theory  and  Applications,  Reed  provides  a  superficial 
account  of  the  separate  stages  that  researchers  most  commonly  include  in  information-processing 
models  (Figure  1).  According  to  Reed,  the  sensory  store  provides  brief  storage  for  information 
in  its  sensory  form.  The  perceptual  information  is  then  filtered  which  results  in  the  blocking  of 
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some  information,  while  other  information  is  aceepted  or  reeognized.  At  the  end  of  the  filtering, 
all  is  lost  unless  a  pattern  is  reeognized  (e.g.,  identifying  a  pattern  as  an  animal  or  a  written  letter 
or  word).  Reed  eontinues  that  the  seleetion  stage  determines  whieh  information  the  person  will 
try  to  remember.  Following  selection,  the  information  is  then  moved  into  short-term  (limited 
capacity  in  amount  and  duration)  or  long-term  memory  (with  unlimited  capacities).  Note  that  the 
stages  are  depicted  in  temporal  order;  however,  the  information  can  flow  in  both  directions  since 
information  in  a  later  stage  can  influence  the  processing  of  new  information  in  earlier  stages. 


Input  Response 


Figure  1.  Stages  of  an  information-processing  model  (adapted  from  Reed,  2004). 

Having  a  basic  understanding  and  conceptual  framework  of  the  cognitive  process  is  of 
benefit  to  anyone  involved  in  the  practical  exercise  of  conveying  information.  According  to 
Ashman  and  Conway  (1997),  it  is  important  to  have  a  basic  understanding  of  the  brain  as  a 
working  cognitive  organizer.  King-Friedrichs  (2001)  writes,  “Teachers  can  use  knowledge  about 
how  the  brain  remembers  to  help  students  retain  [and  recall]  concepts.” 

Learning  style  preferences 

In  a  paper  on  approaches  to  teaching,  Munro  and  Rice-Munro  (2004)  write,  “If  a  topic  is 
important  for  students  to  learn,  present  it  in  a  variety  of  ways  that  will  stimulate  learning. .  .It’s 
clear  that  there  is  no  one  instructional  method  that  will  reach  all  learners;  therefore,  it  is  up  to 
those  designing  and  delivering  the  instruction  to  offer  a  variety  of  approaches.”  The  notion  that 
no  one  teaching  method  will  reach  all  learners  has  emerged  as  the  result  of  extensive  research 
into  individual  student  learning  style  preferences,  specifically,  their  preferred  sense  modality  of 
stimuli  from  which  they  most  effectively  take  in,  process,  and  store  new  information  (Harrison, 
Andrews,  and  Saklofske,  2003;  Cassidy  and  Eachus,  2000;  Dunn,  1983).  Simply  stated,  a 
learning  style  indicates  an  individual’s  preference  for  different  types  of  information,  the  different 
ways  in  which  it  is  perceived,  and  the  rate  at  which  the  information  is  understood  (Felder,  1993). 
A  person’s  learning  style  is  said  to  be  the  combination  of  cognitive,  affective,  and  psychological 
characteristics  that  describe  how  that  individual  interacts  with  his  or  her  environment  (Kratzig 
and  Arbuthnott,  2006).  Thus,  to  ensure  the  effectiveness  of  a  teaching  program,  consideration 
needs  to  be  given  to  the  characteristics  of  the  learner  (Ashman  and  Conway,  1997). 

The  significance  of  learning  styles  gained  standing  as  a  framework  in  the  1970s  in  an 
effort  to  enrich  teaching  methods  and  explain  the  differences  in  how  students  learn  (Brew,  2002; 
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Dunn  and  Dunn,  1979;  Keefe,  1979).  A  diseourse  of  learning  styles  is  not  possible  without  a 
diseussion  of  Kolb’s  Experiential  Learning  Model  (ELM)  (1976).  Kolb’s  ELM  is  a  well 
established  model  that  has  attraeted  mueh  interest  and  appreeiation  (Loo,  2004/2002).  It 
eontinues  to  be  influential  in  the  educational  and  management  fields  (Brew,  2002). 

According  to  Kolb  (1976),  his  model  is  labeled  experiential  to  emphasize  the  important 
role  experience  plays  in  the  learning  process:  “The  core  of  the  model  is  a  simple  description  of 
the  learning  cycle:  how  experience  is  translated  into  concepts,  which  in  turn  are  used  as  guides  in 
the  choice  of  new  experiences.”  Kolb’s  model  is  a  four-stage  cycle  (Ligure  2)  which  describes 
how  learners  see  and  interpret  information:  experience  is  translated  into  concepts,  which  then 
guide  the  choice  of  new  experiences  (Wolfe,  Bates,  Manikowske,  and  Amundsen,  2005).  Kolb 
writes: 


These  observations  are  assimilated  into  a  theory  from  which  new  implications  for  action 
can  be  deduced.  These  implications  or  hypotheses  then  serve  as  guides  in  acting  to  create 
new  experiences.  The  learner,  if  he  is  to  be  effective,  needs  four  different  kinds  of 
abilities:  concrete  experience  (CL),  reflective  observation  (RO),  abstract  onceptualization 
(AC),  and  active  experimentation  (AE). 


Ligure  2.  Kolb’s  Experiential  Learning  Model  (adapted  from  Kolb,  1976). 

Kolb  (1976)  questions  whether  anyone  can  become  highly  skilled  in  all  of  the  four 
abilities.  He  submits  that  learning  requires  abilities  that  are  polar  opposites  and  that  the  learner, 
as  a  result,  must  continually  choose  which  set  of  abilities  he  will  bring  to  bear  in  any  specific 
learning  situation.  Therefore,  he  suggests  that  the  learning  process  possesses  two  primary 
dimensions:  how  individuals  perceive  the  information  (concrete  experience  or  abstract 
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conceptualization)  and  how  they  process  the  information  (using  active  experimentation  or 
refleetive  observation)  (Kolb;  Brew,  2002;  Loo,  2004;  Wolfe,  Bates,  Manikowske,  and 
Amundsen,  2005).  Kolb  eoneluded  that  most  people  emphasize  some  learning  abilities  over 
others  and,  thus,  develop  learning  styles  that  are  based  on  their  heredity,  past  experienee,  and  the 
demands  of  their  present  environment. 

To  eomplement  his  theory,  Kolb  (1976)  developed  a  learning  style  preferenee  instrument, 
the  Learning  Style  Inventory  (LSI),  from  whieh  individual  strengths  and  weaknesses  as  a  learner 
are  measured.  Essentially,  the  instrument  measures  the  individual’s  relative  emphasis  on  the 
four  learning  abilities  listed  earlier.  The  LSI  results  in  the  identifieation  and  labeling  of  four 
learning  styles:  divergers,  eonvergers,  assimilators,  and  aeeommodators.  These  styles  are 
derived  from  and  form  the  four  quadrants  of  Kolb’s  ELM  (Eigure  3). 


Eigure  3.  Kolb’s  Experiential  Eearning  Model  and  four  learning  styles  (adapted  from  Eoo, 
2004). 

Aeeording  to  Kolb  (1976),  divergers  are  best  at  CE  and  RO  and  therefore,  have  great 
imaginative  ability,  ean  view  eonerete  situations  from  many  perspectives,  are  interested  in 
people,  and  are  ereative.  Convergers  are  the  opposite,  with  AC  and  AE  their  dominant  learning 
abilities.  They  are  strong  in  the  praetieal  applieation  of  ideas  and  ean  foeus  on  speeifie  problems 
using  hypothetieal-deduetive  reasoning.  They  are  pragmatie  and  logieal  (Wolfe,  Bates, 
Manikowske,  and  Amundsen,  2005),  have  narrow  technieal  interests,  and  henee,  tend  to 
specialize. 
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Assimilators  possess  robust  AC  and  RO  learning  abilities  whieh  give  them  strength  at 
ereating  theoretieal  models  and  allows  them  to  exeel  in  induetive  reasoning.  Assimilators  have 
eoneern  for  abstraet  eoneepts,  but  are  less  interested  in  the  praetieal  use  of  theories  or  people. 
Accommodators,  on  the  other  hand,  are  the  eonverse  of  the  assimilators.  They  are  best  at  CE  and 
AE,  whieh  means  they  are  best  at  doing  things.  They  aeeomplish  things,  are  risk  takers,  and  are 
the  best  at  adapting  themselves  to  speeifie  situations.  Where  theory  and  faet  disagree, 
aeeommodators  will  most  likely  diseard  the  theory. 

Kolb’s  ESI  has  been  eontroversial  and  eritieized  by  some  as  weak  (Eoo,  2004;  Sadler- 
Smith,  1997)  and  at  worst,  possessing  questionable  validity  and  reliability  (Ereedman  and 
Stumpf,  1978;  for  rebuttal  see  Kolb,  1981;  Brew,  2002).  Studies  have  shown  that  learning  style 
is  not  a  major  determinant  of  learning  preferenee  (Eoo). 

Visual  pereeption 

As  detailed  earlier,  pereeptions  through  the  senses  are  the  first  steps  in  the  eognitive 
proeess.  Normal  individuals  aequire  75%  of  their  knowledge  through  their  sense  of  sight  (U.S. 
Army  Aviation  Center,  2000).  In  the  eontext  of  visual  pereeption,  Dretske  (1995)  makes  a 
distinetion  between  sense  pereeption  and  meaningful  pereeption.  Meaningful  pereeption  refers 
not  to  the  objeets  one  sees,  but  to  how  one  pereeives  them.  This  meaningful  pereeption 
embodies  a  judgment,  belief,  or  reeognition  and  requires  eoneeptual  skills,  thus,  requiring  some 
level  of  visual  cognition.  This  eognitive  quality  of  visual  pereeption  has  been  the  foeus  of  past 
and  eurrent  researeh  and  is  the  foundation  on  whieh  the  IPS,  the  proposed  novel  approaeh  to 
teaehing  utilizing  pietorial  representations,  is  based. 

As  early  as  1886,  researeh  by  James  MeKeen  Cattell  (Carr,  1986)  diseovered  that  people 
eould  reeognize  a  single  eolor  or  shape  in  a  slightly  shorter  time  than  a  word  or  letter;  however, 
it  took  them  longer  to  name  it.  Cattell  explained: 

. .  .this  is  beeause  in  the  ease  of  words  or  letters  that  assoeiation  between  the  idea  and  the 

name  has  taken  plaee  so  often  that  the  proeess  has  beeome  automatie,  whereas,  in  the 

ease  of  eolours  [sie]  and  pietures  we  must  by  voluntary  effort  ehoose  the  name. 

In  1982,  Carr,  MeCauley,  Sperber,  and  Parmelee  (Carr,  1986)  replieated  Cattell’ s 
experiments  and  eame  to  the  same  eonelusions,  adding  further  legitimaey  to  CattelTs 
experiments.  Eindings  from  similar  researeh  by  Potter,  Kroll,  and  Harris  (1979)  on  the  speed  in 
whieh  words  or  pietures  are  named  were  eompatible  with  CattelTs  eonelusions:  that  words  and 
pietures  are  readjust  alike,  exeept  that  words,  being  so  praetieed,  make  the  proeess  of  naming 
them  more  effieient  and  automatie.  If  naming  is  faster  with  words,  then  what  about  the  meaning 
of  the  stimulus  rather  than  its  name?  Researeh  by  Potter  and  Eauleoner  (1975)  answered  that 
interesting  question.  Their  researeh  found  that  although  words  were  named  faster  than  pietures, 
pietures  were  elassified  faster  than  words.  Their  findings  indieate  that  although  words  may  bear 
a  eloser  relationship  to  their  pronuneiations  (at  least  in  a  phonetie  language),  pietures  bear  a 
eloser  relationship  to  their  meanings  (Carr,  1986).  Researeh  has  shown  that  pietures  possessing 
meanings  ean  serve  as  aids  to  memory. 
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Visual  memory 


Early  research  by  Shepard  (1967)  into  recognition  memory  found  that  stimuli  recalled 
from  pictures  were  correctly  recognized  by  98%  of  the  participants  compared  to  90%  for  words 
and  88%  for  sentences.  Comparable  research  by  Nickerson  (1965)  arrived  at  similar 
conclusions.  Nickerson  wrote  that  although  his  experiment  differed  procedurally  in  several 
important  respects,  the  results  of  his  project  substantiated  Shepard’s  finding  “of  exceptionally 
high  recognition  memory  performance  with  pictorial  material.” 

Memory  research  in  the  1960s  by  Haber  (1970)  suggested  that  there  was  one  kind  of 
memory  for  linguistic  information  (words,  numbers,  etc.)  and  another  for  pictorial  information 
(scenes,  pictures,  etc.).  According  to  his  conclusions,  each  kind  of  memory  handled  the  visual 
information  differently.  In  the  case  of  words,  text,  and  numbers,  “the  first  step  of  memory  is  to 
take  the  stimulus  out  of  its  visual,  pictorial  form,  code  the  items,  and  extract  their  meanings” 
(Haber).  Words  are  remembered,  not  as  a  picture  of  their  letters,  but  as  an  idea  or  concept  of  the 
word(s).  This  requires  several  steps  of  encoding  for  proper  storage  and  retrieval.  Evidence  from 
Haber’s  research  indicated  that  this  was  not  the  case  with  pictorial  information.  He  suggested 
that  the  pictorial  image  was  received  and  stored  permanently  in  its  pictorial  form.  Haber  wrote 
that  “the  capacity  of  memory  for  pictures  may  be  unlimited.” 

The  capacity  of  the  human  mind  to  retain  recognition  of  previously  viewed  scenes 
(pictures)  has  been  the  focus  of  numerous  studies  over  the  past  35  years  which  have  repeatedly 
confirmed  the  mind’s  extraordinary  ability  to  discriminate  previously  viewed  objects  and  scenes 
from  hundreds  of  distracters  (Standing,  Conezio,  and  Haber,  1970;  Potter,  1976;  Simons,  1996; 
Hollingworth,  Williams,  and  Henderson,  2001;  Hollingworth  and  Henderson,  2002).  By  and 
large,  during  recognition  studies,  subjects  are  shown  a  number  of  scenes  first.  This  exercise  is 
followed  by  a  second  set  of  scenes  which  contains  previously  viewed  scenes  plus  new  ones. 
Subjects  must  then  identify  the  pictures  that  were  previously  seen  in  the  first  viewing.  The 
studies  have  shown  that  the  participants  perform  exceedingly  well  even  when  thousands  of 
pictures  are  shown.  Hollingworth,  Williams,  and  Henderson  report  that  the  results  of  their  study 
“demonstrate  that  visual  object  representations  are  retained  in  memory  [and  that]  the  study 
supports  a  view  of  scene  perception  in  which  memory  and  perceptual  processing  are  functionally 
integrated.” 

According  to  research  conducted  on  the  brains  of  humans  and  monkeys  (Keysers,  Xiao, 
Eoldiak,  and  Perrett,  2005),  neurons  in  the  temporal  cortex  continue  processing  briefly-flashed 
visual  stimuli  (pictures)  as  if  they  were  still  present.  This  continued  firing  of  neurons  may 
underlie  the  increased  memory  for  pictures  and  icons.  Kosslyn  (2005)  writes  that  the  “visual 
cortex  supports  depictive  representations  during  perception,  not  descriptive  ones... this  is  strong 
evidence  that  imagery  does  not  rely  exclusively  on  the  same  sorts  of  representations  that  underlie 
language.” 

Pictorial  representations 

With  an  understanding  of  visual  cognition  and  with  scientific  evidence  of  the  mind’s 
exceptional  capacity  for  the  recall  of  images,  a  search  for  literature  examining  the  practical 
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application  of  pictorial  systems  was  conducted.  Understanding  the  role  of  pictorial  illustrations 
for  improving  memory  and  verbal  reeall  is  neeessary  in  the  design  and  employment  of  sueh  a 
system. 


In  1993,  Morrell  and  Park  published  a  report  of  a  study  during  which  adults  were 
instructed  to  build  a  3-dimensional  objeet  from  three  types  of  instruetions  (text  only,  illustration, 
or  text  and  illustration).  The  findings  indieated  that  instruetions  eonsisting  of  both  text  and 
illustrations  redueed  errors  compared  to  the  other  formats.  A  study  in  1996  by  Cherry,  Park, 
Frieske,  and  Smith,  found  that  pictorial  illustrations  had  a  positive  effect  on  younger  and  older 
adults’  reeall  of  adjeetives.  A  similar  study  by  Cherry,  Dokey,  Reese,  and  Brigman  (2003), 
tested  the  effeets  of  verbal  and  pietorial  illustrations  on  younger  and  older  adults’  reeall  of  the 
eontent  of  short  sentenees.  Positive  effects  of  pictorial  illustrations  were  observed  and  the  data 
were  elear  that  both  groups  benefited  from  the  mere  presenee  of  pietures.  The  authors  write, 
“With  an  eye  toward  everyday  memory  praetiees,  this  aspeet  of  the  data  suggests  that  pietorial 
illustrations  should  be  ineluded  in  instruetional  formats  that  are  designed  to  promote  learning  and 
retention  of  written  materials  in  everyday  life  for  both  younger  and  older  adults.” 

There  are  aspeets  of  pietorial  representations,  however,  that  must  be  eonsidered  when 
ineluding  pietorial  representations  in  learning  material.  Pietorial  representations  must  be 
supported  by  substantive  explanations  and/or  instruction.  Watkins,  Miller,  and  Brubaker  (2004) 
note  that  most  students  tend  to  rely  solely  on  illustrations,  when  they  are  present,  to  the  exclusion 
of  captions  or  explanatory  text.  This  propensity  to  ignore  the  supporting  text  results  in  the 
eonstruetion  of  their  own  interpretations,  thus,  leading  to  miseoneeptions  of  the  information.  In 
faet,  Morgan  (2005)  discovered  that  participants  in  a  study  of  magazine  advertisements  do  infer  a 
eonsistent  set  of  claims,  but  also  arrive  at  multiple  meanings  (interpretations  and  eoneeptions) 
within  eaeh  advertisement.  With  this  in  mind,  preeautions  should  be  taken  to  ensure  that 
pietorial  representations  are  pereeived  and  understood  as  intended. 

IPS  symbols 

When  designing  symbols,  especially  those  purporting  to  be  intuitive  as  with  the  IPS, 
many  of  the  eonstruets  and  guidelines  utilized  by  eomputer  software  designers  are  useful  and 
relevant.  Designers  devise  symbols  (visual  representations)  as  interfaees  between  the  human  and 
the  maehine  ereating  a  language  of  symbols  and  metaphors  (Szewezyk,  2003).  Such  symbols  are 
referred  to  as  graphic  user  interfaces  or  GUI  (pronounced  GOO-ee).  Ambler  (2000,  quoted  by 
Szewezyk)  notes  that  the  GUI  strueture  includes  ieons,  palettes,  dialogues,  and  cursor  shapes. 
Aeeording  to  Ambler,  the  layout  of  the  symbols  influenees  the  way  the  user  ean  interpret  it  and 
the  relations  between  pieees  of  the  GUI  strueture  add  some  new  meanings  to  the  visual 
languages  of  the  interfaces. 

Referring  to  [Worldwide]  Web  interfaee  design,  but  still  applieable  to  the  proposed  IPS, 
Dr.  R.H.  Miller  (n.d.),  Projeet  Manager  for  Bell  Communieations  Researeh,  writes  that  when 
designing  ieons,  it  is  important  ask  several  key  questions: 

•  Is  the  design  simple? 

•  Is  the  meaning  clear  for  the  intended  audienee? 
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•  Does  any  action  represented  in  the  icon  standout?  (emphasize  the  action  since  it  is  hard  to 
present  motion  in  an  icon) 

•  Are  you  using  consistent  and  appropriate  metaphors? 

•  If  “yes”  can  be  answer  to  all  of  the  questions  above,  then  ask,  “Is  the  icon  unique?” 

In  Szewczyk’s  2003  study  of  computer-aided  design  (CAD)  students,  he  drew  some 
inferences  from  the  results  concerning  the  students’  perception  of  the  CAD  GUI; 

1 .  If  an  unknown  icon  looked  similar  to  a  well-known  icon,  students  supposed  that 
the  unknown  icon  represented  the  same  function.  The  students  used  empirical 
knowledge  and  demonstrated  a  strong  desire  to  associate  unknown  icons  with 
those  that  were  well  known. 

2.  If  an  icon  was  a  picture  of  a  real  thing,  students  supposed  that  it  could  be  used  to 
create  or  modify  the  virtual  representation  of  such  a  thing.  Using  obvious  and 
simple  metaphors  improves  the  association  to  the  real  thing. 

3.  Abstract  icons  misled  students  and  created  problems  wherein  students’ 
perceptions  of  the  icons  were  extremely  individualistic.  Interestingly, 
inexperienced  CAD  students  (second-semester)  tended  to  guess  more  at  the  icons’ 
meanings  and  gave  many  incorrect  answers,  while  the  more  experienced  students 
(sixth-semester)  avoided  guessing  and  did  not  answer  or  comment  as  to  their 
meanings. 

4.  In  keeping  with  Number  3  above,  the  inexperienced  students  tried  to  make  sense 
out  of  any  icon,  while  the  more  experienced  students,  having  a  fundamental 
knowledge  of  icons,  did  not  seem  to  see  the  necessity  in  learning  or  guessing  at 
every  icon. 

5.  The  layout  of  GUI  symbols  can  affect  the  understanding  of  each  symbol. 
Inexperienced  students,  in  their  effort  to  make  sense  out  of  any  icon,  had  a  better 
sense  of  the  context  in  which  the  icons  appeared.  In  contrast,  experienced 
students,  who  typically  ignored  unknown  icons,  failed  to  become  aware  of  the 
context  in  which  the  icons  appeared.  Their  focus  remained  on  the  known  icons. 

The  symbols  (Appendix  A)  which  make  up  the  IPS  were  developed  consistent 
with  the  above  guiding  principles. 


Conceptual  framework 

In  order  to  test  whether  the  form  in  which  information  is  presented  (taught)  results  in  any 
change  in  retention  and  therefore,  the  recall  of  the  information,  the  conceptual  framework 
consists  of  a  series  of  entities  (Figure  4).  The  visual  form  in  which  the  information  is  presented 
affects  the  way  in  which  it  is  perceived  by  the  student.  Theory  suggests  that  pictures  are 
perceived  differently  than  text.  Such  perceptions  produce  a  visual  memory  that,  if  meaningful, 
produces  visual  cognition.  Cognition  or  learning  occurs  when  the  information  is  processed 
(encoded)  and  moved  from  short-term  memory  to  long-term  memory.  Mnemonic  strategies, 
such  as  those  representing  information  in  pictorial  form,  have  demonstrated  the  capacity  to 
facilitate  such  encoding  and  studies  indicate  considerable  success  when  using  such  strategies  in 
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the  retention  and  reeall  of  the  information  learned.  Thus,  improved  memory  performance  is  the 
ultimate  eonstruet  of  the  proposed  study. 
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Figure  4.  Coneeptual  framework  of  the  study. 

Researeh  hypotheses 

Based  on  the  theoretieal  and  empirieal  researeh  regarding  the  eonstruet  of  this  study,  the 
proposed  study  will  test  the  following  researeh  hypotheses: 

Hoi:  Based  on  performanee  test  seores,  there  is  no  differenee  in  the  demonstrated  retention  and 
reeall  of  aviation  emergeney  proeedures  between  the  traditional  and  novel  teaehing  methods  in 
the  overall  sample  population. 

Hai:  Based  on  performanee  test  seores,  there  is  a  differenee  in  the  demonstrated  retention  and 
reeall  of  aviation  emergeney  proeedures  between  the  traditional  and  novel  teaehing  methods  in 
the  overall  sample  population. 

H02:  Based  on  performanee  test  seores,  there  is  no  differenee  between  those  taught  using  the 
traditional  method  and  those  taught  using  the  IPS  method  with  respeet  to  experienee  levels. 

Ha2:  Based  on  performanee  test  seores,  there  is  a  differenee  between  those  taught  using  the 
traditional  method  and  those  taught  using  the  IPS  method  with  respeet  to  experienee  levels. 

H03:  Based  on  performanee  test  seores,  there  is  no  differenee  between  those  taught  using  the 
traditional  method  and  those  taught  using  the  IPS  method  with  respeet  to  individual  learning 
style  preferenees. 

Has:  Based  on  performanee  test  seores,  there  is  a  differenee  between  those  taught  using  the 
traditional  method  and  those  taught  using  the  IPS  method  with  respeet  to  individual  learning 
style  preferenees. 

H04:  Reeall  performanee  is  not  related  to  the  manner  in  whieh  the  IPS  is  subjeetively  assessed. 
Ha4:  Reeall  performanee  is  related  to  the  manner  in  whieh  the  IPS  is  subjeetively  assessed. 
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Hos:  Based  on  opinion  data,  individual  learning  style  preferenees  do  not  have  an  effeet  on 
aviators’  subjeetive  assessment  of  the  IPS  (using  pietorial  mnemonies)  as  an  aid  in  memorizing 
textual  aviation  emergeney  proeedures. 

Has:  Based  on  opinion  data,  individual  learning  style  preferences  do  have  an  effect  on  aviators’ 
subjective  assessment  of  the  IPS  (using  pictorial  mnemonics)  as  an  aid  in  memorizing  textual 
aviation  emergency  procedures. 

Ho6:  Based  on  opinion  data,  experience  levels  have  no  effect  on  aviators’  subjective  assessment 
of  the  IPS  (using  pictorial  mnemonics)  as  an  aid  in  memorizing  textual  aviation  emergency 
procedures. 

Ha6:  Based  on  opinion  data,  experience  levels  do  have  an  effect  on  aviators’  subjective 
assessment  of  the  IPS  (using  pictorial  mnemonics)  as  an  aid  in  memorizing  textual  aviation 
emergency  procedures. 

Ho?:  Based  on  opinion  data,  the  novel  IPS  was  not  positively  assessed  by  the  majority  of  those  in 
the  treatment  groups  as  a  useful  pictorial  mnemonic  for  remembering  textual  aviation  emergency 
procedures. 

Ha?:  Based  on  opinion  data,  the  novel  IPS  was  positively  assessed  by  the  majority  of  those  in 
the  treatment  groups  as  a  useful  pictorial  mnemonic  for  remembering  textual  aviation  emergency 
procedures. 

Hog:  Based  on  symbol  recognition  test  scores,  there  is  no  difference  in  the  ability  to  easily 
recognize  and  remember  the  IPS  symbols  from  Day  1  to  Day  7. 

Has:  Based  on  symbol  recognition  test  scores,  there  is  a  difference  in  the  ability  to  easily 
recognize  and  remember  the  IPS  symbols  from  Day  1  to  Day  7. 

Hog:  Based  on  posttest  performances,  there  is  no  association  between  committed  study  time  from 
Day  1  to  Day  7  and  test  performances. 

Ha9:  Based  on  posttest  performances,  there  is  an  association  between  committed  study  time 
from  Day  1  to  Day  7  and  test  performances. 

Research  methodology 

Research  designs 

Primarily,  this  study  tested  the  merits  of  the  IPS  through  a  true  experimental  design:  a 
pretest-posttest  control  group  design  (Leedy  and  Ormrod,  2001).  The  design  was  applied  to 
three  distinct  membership  groups  based  on  their  experience  levels  and,  thus,  their  exposures  to 
the  traditional  training  method.  This  was  desired  in  an  effort  to  gain  further  insight  into  the 
applicability,  perceptions,  and  assessments  of  the  IPS  based  on  pilots’  experience  levels.  The 
first  membership  group  (MG  1)  was  comprised  of  instructor  pilots  considered  highly  experienced 
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in  traditional  emergency  procedure  training.  The  second  membership  group  (MG  2)  was 
comprised  of  student  pilots  with  minimal  experience  in  traditional  emergency  procedures 
training.  Members  of  MG  2  were  student  pilots  undergoing  training  in  the  U.S.  Army’s  UH-60 
Black  Hawk  helicopter  aircraft  qualification  course.  The  third  group  (MG  3)  was  comprised  of 
students  waiting  to  start  flight  school  (naive  to  emergency  procedure  training).  The  intent  of 
testing  this  group  was  to  derive  insight  into  which  of  the  two  training  methods,  the  traditional  or 
novel,  was  most  effective  in  causing  learning,  retention,  and  recall  when  presenting  the 
information  to  students  who  were  naive  to  any  previous  aviation  emergency  procedures  training. 

Consistent  with  the  experimental  design,  an  experimental  or  treatment  group  and  a 
control  group  was  selected  by  random  assignment  within  each  membership  group.  On  Day  1, 
the  knowledge  of  the  treatment  group  was  pretested  (Emergency  Procedures  Knowledge  Test, 
Appendix  B),  subjected  to  the  treatment  (trained  in  the  use  of  the  novel  IPS),  and  then  posttested 
with  the  same  test.  The  control  group  on  Day  1  was  pretested,  received  training/review  of  the 
emergency  procedures  in  the  traditional  manner,  and  posttested.  The  treatment  and  control 
groups  of  each  membership  group  were  posttested  seven  days  later.  Each  posttest  was  intended 
to  measure  the  learning,  retention,  and  recall  of  the  treatment  group  compared  to  that  of  the 
equivalent  control  group.  Eigure  5  depicts  the  paradigm  for  the  design. 


Random 

Assignment 

Group 

Time 

Dav  1  - ► 

Day  7 

Treatment 

Group 

Pretest 

Novel  Training 
(Treatment) 

Posttest 

Posttest 

Control 

Group 

Pretest 

Traditionally  trained 

Posttest 

Posttest 

Eigure  5.  Pretest-Posttest  control  group  design  (Eeedy  and  Ormrod,  2001). 

In  order  to  test  the  IPS’s  merit  as  a  mnemonic  strategy,  and  the  intuitiveness  and 
memorable  qualities  of  the  IPS  symbols  themselves,  the  treatment  groups  of  each  membership 
group  received  two  additional  tests:  the  Subjective  Assessment  Survey  (Appendix  D)  and  the 
Symbol  Recognition  Test  (Appendix  E).  These  tests  were  administered  immediately  following 
treatment  on  Day  1  and  then  again  Day  7.  The  design  is  that  of  an  equivalent  time-samples 
design  (Eeedy  and  Ormrod,  2001)  and  is  depicted  in  Eigure  6.  Note  the  absence  of  the  pretest. 
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Eigure  6.  Equivalent  time-samples  design:  MGs  1,  2,  and  3  Treatment  Groups  (Eeedy  and 
Ormrod,  2001). 
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Signed  informed  eonsents  were  obtained  from  eaeh  participant  prior  to  his  or  her 
participation.  All  identifying  data  were  separated  from  test  data  and  were  stored  in  a  locked  safe 
in  the  USAARL  Science  Information  Center  for  a  minimum  of  three  years. 

Participants  and  setting 

As  described  above,  the  study  population  consisted  of  three  distinct  membership  groups: 
highly-experienced  instructor  pilots  (MG  1),  minimally-experienced  student  pilots  (MG  2),  and 
naive  students  waiting  to  start  flight  school  (MG  3).  All  participants  were  recruited  from 
personnel  assigned  to  Fort  Rucker,  Alabama,  and  were  identified  through  local  advertisement 
and  solicitation.  Fort  Rucker  is  the  location  of  the  U.S.  Army  Aviation  Warfighting  Center 
(USAAWC).  All  training  and  testing  for  this  study  occurred  in  the  aviation  training  classrooms 
at  Lowe  Army  Heliport  or  in  a  USAARL  meeting  room,  both  located  at  Fort  Rucker,  Alabama. 

Sample  size 

The  sample  size  required  to  achieve  a  sufficient  power  to  examine  the  potential  of  the  IPS 
strategy  was  performed  using  the  DSS  Research  calculator,  an  online  power  calculation  tool 
(DSS  Research,  n.d.).  The  power  calculation  was  conducted  specifying  a  desired  power  of  .80, 
an  a  level  of  .05,  and  a  large  effect  size  (d  =  1 .00).  The  effect  size  of  1 .00  was  based  on  the 
results  of  a  study  by  Carney  and  Levin  (2003).  Carney  and  Levin’s  work  examined  the  effects  of 
training  undergraduate  students  in  the  use  of  mnemonic  strategies  (pictorial  representations)  on 
their  memory  and  recall  of  unfamiliar  hierarchical  information,  which  was  sufficiently  similar  to 
the  objectives  of  this  study.  With  these  specifications  and  Carney  and  Levin’s  findings  as  a  basis 
for  the  power  calculation,  30  participants  per  membership  group  (15  per  treatment  group  and  15 
per  control  group)  were  required  to  ensure  an  80%  chance  of  rejecting  a  false  null  hypothesis. 
Hence,  this  study  required  a  total  n  size  of  90,  30  per  membership  group. 

Data  collection  tools 
Emergency  Procedures  Knowledge  Test 

The  Emergency  Procedures  Knowledge  Test  (Appendix  B)  for  all  membership  groups 
consisted  of  a  15 -question  fill-in- the-b lank  written  examination  covering  17  selected  emergency 
procedures  listed  in  the  UH-60  Black  Hawk  Operator’s  Manual  (Headquarters,  Department  of 
the  Army,  2003).  The  test  had  a  total  of  25  answers  with  each  answer  worth  four  points;  thus, 
the  final  test  scores  ranged  from  0%  to  100%  correct  and  were  scored  using  the  answer  sheet  in 
Appendix  C. 

VARK  Teaming  Styles  Questionnaire 

The  VARK  Teaming  Styles  Questionnaire  (n.d.)  (Appendix  H)  provided  a  profile  of 
user  learning  preferences:  visual  (information  displayed  in  charts,  graphs,  symbols),  aural  (heard, 
spoken),  read/write  (words,  text),  kinesthetic  (experience,  practice)  and  multimodal  (combination 
of  several  styles).  (Permission  to  use  the  VARK  questionnaire  was  granted  by  Mr.  Neil  D. 
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Fleming,  Christchurch,  New  Zealand,  holder  of  the  copyright,  on  August  27,  2005.)  VARK 
categories  were  determined  according  to  the  scoring  procedures  in  Appendix  I. 

Subjective  Assessment  Survey 


The  Subjective  Assessment  Survey  (Appendix  D)  consisted  of  a  series  of  questions  aimed 
at  determining  the  merits  of  the  IPS  as  a  mnemonic  strategy  based  on  the  opinions  of  the  users. 
The  responses  were  made  on  a  five-point  Likert  scale  (Wikipedia,  n.d.)  ranging  from  strongly 
disagree  to  strongly  agree.  Solicitation  for  additional  comments  was  made.  Scoring  was 
performed  per  the  instructions  in  Appendix  E. 

Symbol  Recognition  Test 

The  Symbol  Recognition  Test  (Appendix  F)  was  a  20-symbol,  fill-in-the-blank  test  that 
was  intended  to  gauge  the  intuitiveness  and  memorable  quality  of  the  IPS  symbols.  The  test  had 
a  total  of  20  answers  with  each  answer  worth  5  points,  thus,  the  final  test  scores  ranged  from  0% 
to  100%  correct.  Scoring  was  performed  using  the  answer  sheet  in  Appendix  G. 

Study  Experience  Report 

The  Study  Experience  Report  (Appendix  J)  was  added  to  the  study  on  the 
recommendation  of  the  USAARL  Science  Program  Director  to  gauge  the  study  efforts  of  the 
sample  population.  The  report  collected  an  estimate  of  the  time  spent  studying  the  emergency 
procedures  presented  as  part  of  this  research,  with  and  without  using  the  IPS  symbols  as 
appropriate  to  the  assigned  group. 


Research  variables 

Research  variables  were  derived  from  each  of  the  data  collection  instruments  described 
above  and  employed  as  independent  variables  (IV),  dependent  variables  (DV),  or  covariates 
(CV).  Table  1  presents  the  research  variables  and  their  functions  in  addressing  each  research 
question  (RQ). 
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Table  1. 

Research  variables. 


RQ 

Variables 

Variable 

function 

Variable  Type/ 

Level  of  Measurement 

1 

Method  of  Training 

IV 

Categorical/N  ominal 

Knowledge  Test  Change  Scores 

DV 

Continuous/Ratio 

2 

Method  of  Training 

IV 

Categorical/N  ominal 

Experience  Level 

IV 

Categorical/N  ominal 

Knowledge  Test  Change  Scores 

DV 

Continuous/Ratio 

3 

Method  of  Training 

IV 

Categorical/N  ominal 

Learning  Style  Preference 

IV 

Categorical/N  ominal 

Knowledge  Test  Change  Scores 

DV 

Continuous/Ratio 

■ 

Subjective  Assessment 

DV  (predictor) 

CategoricaLOrdinal 

Knowledge  Test  &  Change  Scores 

DV  (outcome) 

Continuous/Ratio 

5 

Learning  Style  Preference 

DV  (predictor) 

Categorical/N  ominal 

Subjective  Assessment 

DV  (outcome) 

CategoricaLOrdinal 

6 

Experience  Level 

DV  (predictor) 

Categorical/N  ominal 

Subjective  Assessment 

DV  (outcome) 

CategoricaLOrdinal 

7 

Subjective  Assessment 

Descriptive 

CategoricaLOrdinal 

8 

Experience  Level 

IV 

Categorical/Nominal 

Symbol  Recognition  Scores 

DV 

Continuous/Ratio 

9 

Study  Experience 

DV  (predictor) 

Continuous/Ratio 

Knowledge  Test  Change  Scores 

DV  (outcome) 

Continuous/Ratio 

Symbol  Recognition  Change  Scores 

DV  (outcome) 

Continuous/Ratio 

Note:  RQ  =  Research  Question;  IV  =  independent  variable;  DV  =  dependent  variable. 


Procedures 

Each  potential  participant  was  briefed  on  the  objectives  of  the  study.  If  continued 
participation  was  indicated,  each  participant  was  given  adequate  time  to  review  and  understand 
all  the  information  in  the  informed  consent  form  before  agreeing  to  take  part  in  the  research 
study.  After  signing  the  informed  consent,  the  participants  were  randomly  assigned  to  either  a 
treatment  group  or  a  control  group  and  assigned  to  the  appropriate  experience  level  membership 
group:  MG  1,  MG  2,  or  MG  3  (30  participants  per  membership  group  for  a  total  of  90).  The 
importance  of  not  discussing  any  aspects  of  the  study  with  anyone  other  than  those  within  their 
assigned  membership  groups  was  stressed.  Participants  were  instructed  on  what  date  to  report 
for  Day  1 .  Each  group  (treatment  or  control)  was  managed  on  different  days  and  times  of  the 
week  in  order  to  eliminate  any  interaction  between  groups  of  different  experience  levels  (MGs). 


21 


Group  reporting  times  remained  flexible  in  order  to  aecommodate  partieipant  flight  training 
and/or  work  sehedules.  In  order  to  avoid  any  subjeet  bias  possibly  introdueed  by  the  words 
“novel”  and  “traditional,”  the  terms  “Method  1”  and  “Method  2”,  respeetively,  were  used 
whenever  interacting  with  the  subjects. 


Data  collection 


(6) 

On  Day  1  for  a  given  group,  each  participant  completed  a  VARK  Learning  Preference 
Questionnaire  (Appendix  H).  Following  that,  each  group  received  a  general  information  briefing 
(Appendix  K)  and  was  administered  the  Emergency  Procedures  Knowledge  Test  (Appendix  B) 
as  a  pretest. 

Next,  depending  on  their  group  of  assignment,  members  were  provided  training  of 
aviation  emergency  procedures  by  either  Method  1  (novel)  or  Method  2  (traditional).  The 
training  consisted  of  classroom  instruction  supported  by  PowerPoint  presentations  during  which 
a  UH-60  aircraft/cockpit  orientation/review  was  presented  (Appendix  L).  This  orientation/ 
review  was  followed  by  training  on  the  17  selected  emergency  procedures  (Appendix  M  or  N, 
depending  on  group  assignment)  listed  in  the  UH-60  Black  Hawk  Operator’s  Manual 
(Headquarters,  Department  of  the  Army,  2003).  It  is  important  to  realize  that  non-underlined 
steps  in  the  Operator’s  Manual  are  not  routinely  memorized,  only  the  underlined  steps. 

However,  as  a  condition  of  this  research,  non-underlined  steps  of  the  selected  emergency 
procedures  were  required  to  be  memorized.  This  ensured  that  even  experienced  aviators  had  to 
apply  some  memorization  techniques  (their  own  or  the  IPS  depending  on  group  assignment) 
during  their  participation  in  this  study. 

At  the  completion  of  the  training,  the  Emergency  Procedures  Knowledge  Test  (Appendix 
B)  was  administered  again  as  a  check  on  learning  (posttest.  Day  1).  In  addition,  members  of  the 
treatment  group  (receiving  Method  1)  were  administered  the  Symbol  Recognition  Test 
(Appendix  E)  and  were  provided  with  a  paper  copy  of  Appendix  A,  Intuitive  Pictorial  System 
Symbols  and  Rules,  for  additional  review  and  study.  Members  of  the  groups  received  a  paper 
copy  of  their  appropriate  PowerPoint  presentation  slides  (Appendix  M  or  N).  On  Day  7,  the 
same  Emergency  Procedures  Knowledge  Test  (Appendix  B)  was  administered  as  a  second 
posttest.  Eollowing  the  posttest,  a  Subjective  Assessment  Survey  (Appendix  D)  was 
administered  to  the  members  of  the  treatment  groups  in  order  to  gather  opinion-based  data 
regarding  the  merits  of  the  novel  IPS.  The  Symbol  Recognition  Test  (Appendix  E)  was  re¬ 
administered  to  the  treatment  groups  only  and  all  groups  were  asked  to  estimate  time  spent 
studying  over  the  previous  week  (Appendix  J). 

The  preceding  data  collection  procedures  are  summarized  in  Table  2. 
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Table  2, 

Event  and  data  eolleetion  summary. 


Treatment 

Groups 

Control 

Groups 

Screening  Day 

-  Briefing  of  Study  Objectives 

-  Informed  Consents  Completed 

-  Random  Group  Assignments  Made 

-  Establish  Group  Reporting  Dates  &  Times 

All  participants 

Day  1 

-  VARK  Teaming  Preference  Questionnaire 

X 

X 

-  General  Information  Briefing 

X 

X 

-  Emergency  Procedures  Knowledge  Test  (pretest) 

X 

X 

-  Instmctional  PowerPoint  Presentation 

t 

t 

-  Emergency  Procedures  Knowledge  Test  (posttest  1) 

X 

X 

-  Symbol  Recognition  Test 

X 

-  Hand  out  paper  copies  of  instmctional  material 

t 

Day  7 

-  Emergency  Procedures  Knowledge  Test  (posttest  2) 

X 

X 

-  Subjective  Assessment  Survey 

X 

-  Symbol  Recognition  Test 

X 

-  Study  Experience  Report 

X 

X 

Note:  t  =  with  IPS  instruetions/symbols;  J  =  without  IPS  instruetions/symbols. 


Testing  and  analysis  of  the  data 

The  testing  and  analyses  of  the  data  for  this  study  were  eondueted  using  SPSS®  12.0  with 
statistieal  signilieance  set  at  the  .05  level  and  eonfidence  intervals  at  95%.  The  data  eolleetion 
effort  resulted  in  the  minimum  number  of  participants  required  per  the  power  analysis  (a 
minimum  of  15  in  each  group).  The  final  total  was  93  as  depicted  in  Table  3. 
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Table  3. 

Sample  population  totals. 


MG  1 

Highly-experienced 

Pilots 

MG  2 

Minimally-experienced 
Student  Pilots 

MGS 

Naive 

Group 

Treatment 

Control 

Treatment 

Control 

Treatment 

Control 

15 

15 

17 

15 

16 

15 

Note:  MG  =  M 

lembership  Group 

The  results  are  presented  in  the  order  of  researeh  hypotheses  and  assoeiated  research 
questions  presented  in  Chapters  2  and  3. 

Hypotheses  Hoi  and  Hai 

To  address  Hypotheses  Hoi  and  Hai  and  answer  whether  the  employment  of  the  IPS 
would  demonstrate  improved  retention  and  recall  of  emergency  procedures  over  that  of  the 
traditional  teaching  method  in  the  overall  sample  population,  two  comparisons  were  made  of  the 
Emergency  Procedures  Knowledge  Test  change  scores:  1)  Day  1  Pretest  and  Day  1  Posttest,  and 
2)  Day  1  Posttest  and  Day  7  Posttest.  An  initial  exploration  of  the  test  score  data  provided  the 
descriptive  statistics  presented  in  Table  4. 
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Table  4, 

Emergency  Procedures  Knowledge  test  descriptive  statistics  for  total  sample  population. 


Treatment/ 

Control 

n 

Mean 

Standard 

Deviation 

Std.  Error 
Mean 

Day  1  Emergency  Procedures 
Knowledge  Pretest 

Treatment 

48 

56.25 

37.318 

5.386 

Control 

45 

47.29 

33.589 

5.007 

Day  1  Emergency  Procedures 
Knowledge  Poshest 

Treatment 

48 

65.00 

28.945 

4.178 

Control 

45 

66.49 

22.471 

3.350 

Day  7  Emergency  Procedures 
Knowledge  Poshest 

Treatment 

48 

72.83 

22.048 

3.182 

Control 

45 

73.51 

20.276 

3.023 

Day  1  Pretest  /  Day  1  Poshest 
Change  Scores 

Treatment 

48 

8.75 

12.985 

1.874 

Control 

45 

19.20 

19.551 

2.915 

Day  1  Poshest  /  Day  7  Poshest 
Change  Scores 

Treatment 

48 

7.83 

17.754 

2.563 

Control 

45 

7.02 

15.682 

2.338 

During  the  initial  data  exploration,  a  Shapiro-Wilk  test  of  normality  was  performed. 
Note  that  a  Shapiro-Wilk  test  resulting  in  a  low  significance  value  (less  than  .05)  indicates  that 
the  distribution  of  the  data  differs  significantly  from  a  normal  distribution.  The  Shapiro-Wilk 
tests  of  the  Day  1  Pre -/Poshest  change  scores  for  the  control  group  and  of  the  Day  1/Day  7 
Poshest  change  scores  for  the  treatment  group  showed  that  these  data  were  not  normally 
distributed  [W(45)  =  .913, p  =  .003  and  JV(48)  =  .891, p  =  .001,  respectively].  Attempts  to 
transform  the  data  did  not  change  its  condition.  Therefore,  the  nonparametric  Mann- Whitney  U 
test  was  used  in  lieu  of  the  originally-planned  independent-samples  t  test  as  the  test  for 
comparing  the  performance  of  the  treatment  and  control  instructional  methods. 

Between  methods 


In  the  first  comparison  (Day  1  Pretest  to  the  Day  1  Poshest),  the  results  indicated  that  in 
the  overall  sample  population,  there  was  a  significant  performance  difference  between  the 
treatment  and  control  instructional  methods.  The  control  method  demonstrated  a  significantly 
greater  average  improvement  (U  =  734.50, =  .008)  in  performance  than  did  the  treatment 
method  (19.2  vs.  8.75,  respectively).  Therefore,  for  this  first  comparison,  the  results  indicate  a 
rejection  of  the  null  hypothesis  (Hoi)  and  support  for  the  alternative  (Hai).  The  results  answer 
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the  research  question  by  revealing  that  the  use  of  the  IPS  did  not  demonstrate  improved  retention 
and  recall  of  emergency  procedures  over  that  of  the  traditional  teaching  method  in  the  overall 
sample  population. 

In  the  second  comparison,  the  Mann- Whitney  U  test  showed  that  there  were  no 
significant  differences  in  the  performance  changes  between  groups  (aggregate  treatment  and 
control  instructional  methods)  from  Day  1  to  Day  7  (U  =  1051.00,/>  =  .823).  In  this  case,  the 
null  hypothesis  (Hoi)  was  found  to  be  true. 

Within  methods 


A  check  of  within-method  performances  using  the  nonparametric  Wilcoxon’s  Signed- 
Rank  test  (alternative  to  paired  samples  t  test)  revealed  that  each  method  group  made  statistically 
significant  improvements  in  their  posttest  performances  from  Day  1  to  Day  7.  Within  the 
treatment  instructional  method,  an  improvement  from  a  mean  of  65.00  to  72.83  {p  =  .007)  was 
noted,  while  the  control  instructional  method  improved  from  a  mean  of  66.49  to  73.51  {p  =  .007). 

Hypotheses  H02  and  Ha2 

To  test  the  Hypotheses  H02  and  Ha2,  a  Kruskal- Wallis  test  (nonparametric  alternative  to 
an  ANOVA)  was  employed  to  answer  whether  there  was  a  difference  in  the  recall  performance 
between  those  taught  using  the  traditional  method  and  those  taught  using  the  IPS  method  with 
respect  to  experience  levels  (MGs).  This  was  followed  by  a  Median  test  which  is  used  to 
determine  whether  any  patterns  represented  significant  group  differences.  Detailed  descriptive 
statistics  for  the  EP  Knowledge  test  scores  and  change  scores  are  presented  in  Tables  5  and  6. 
Individual  graphs  (Figures  0-1  through  0-3)  for  the  EP  Knowledge  test  and  change  scores  by 
membership  group  are  available  in  Appendix  O.  Figure  7  presents  a  summary  of  the  mean 
scores  for  each  of  the  EP  Knowledge  tests  by  membership  group. 
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Table  5. 

Emergency  Procedures  Knowledge  test  score  descriptive  statistics  by  membership  group 

Membership  Treatment/  Std. 

Group _ Control _ n  Mean  Deviation  Min  Max 


Day  1 

EP 

MG  1 

Treatment 

15 

80.00 

17.370 

40 

100 

Knowledge 

Pretest 

Control 

15 

76.80 

14.674 

48 

96 

MG  2 

Treatment 

17 

81.65 

11.230 

64 

100 

Control 

15 

59.47 

16.621 

32 

88 

MG  3 

Treatment 

16 

7.00 

8.066 

0 

28 

Control 

15 

5.60 

10.006 

0 

40 

Day  1 

EP 

MG  1 

Treatment 

15 

82.67 

16.189 

52 

100 

Knowledge 

Posttest 

Control 

15 

84.80 

15.580 

56 

100 

MG  2 

Treatment 

17 

83.29 

12.429 

60 

100 

Control 

15 

68.00 

19.654 

40 

100 

MGS 

Treatment 

16 

29.00 

11.911 

■I 

44 

Control 

15 

46.67 

13.238 

16 

64 

Day  7 

EP 

MG  1 

Treatment 

15 

88.00 

9.798 

64 

100 

Knowledge 

Posttest 

Control 

15 

84.53 

15.108 

60 

100 

MG  2 

Treatment 

17 

80.47 

15.025 

44 

100 

Control 

15 

79.20 

12.394 

64 

96 

MGS 

Treatment 

16 

50.50 

19.093 

24 

88 

Control 

15 

56.80 

21.123 

20 

88 

Note:  MG  =  Membership  Group;  Min  =  minimum;  Max  =  maximum. 
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Table  6. 

Emergency  Procedures  Knowledge  test  change  score  descriptive  statistics  by  membership  group. 


Membership 

Group 

Treatment/ 

Control 

Mean 

n 

Std. 

Deviation 

Min 

Max 

Day  1  Pretest 
to  Day  1 
Posttest 

Change 

Scores 

MG  1 

Treatment 

2.67 

15 

8.902 

-16 

20 

Control 

8.00 

15 

7.856 

-12 

20 

MG  2 

Treatment 

1.65 

17 

7.491 

-8 

16 

Control 

8.53 

15 

10.350 

-12 

24 

MG  3 

Treatment 

22.00 

16 

10.633 

■I 

40 

Control 

41.07 

15 

16.246 

■I 

60 

Day  1 

Posttest  to 

Day  7 

Posttest 

Change 

Scores 

MG  1 

Treatment 

5.33 

15 

14.316 

-8 

44 

Control 

-.27 

15 

7.324 

-12 

12 

MG  2 

Treatment 

-2.82 

17 

10.273 

-20 

16 

Control 

11.20 

15 

13.455 

-12 

32 

MGS 

Treatment 

21.50 

16 

18.698 

H 

56 

Control 

10.13 

15 

21.267 

-32 

44 

Mote:  MG  =  Membership  Group;  Min  =  minimum;  Max  =  maximum. 
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100 


Day  1  EP 

Knowledge  Pretest 
Day  1  EP 

Knowledge  Posttest 
Day  7  EP 

Knowledge  Posttest 


MG1  IPS  MG1  MG2  IPS  MG2  MGS  IPS  MGS 

Traditional  Traditional  Traditional 

Membership  Group 

Figure  7.  Summary  of  Emergency  Procedures  Knowledge  test  score  means  by  membership 
group.  (MG  =  membership  group;  IPS  =  Intuitive  Pictorial  System) 

In  an  analysis  of  these  data,  the  Kruskal- Wallis  test  detected  that  significant  differences 
existed  in  the  changes  in  EP  Knowledge  test  scores  from  Day  1  Pretest  to  Day  1  Poshest  and 
from  Day  1  Poshest  to  Day  7  Poshest  with  respect  to  experience  levels  (Table  7),  hence, 
rejecting  the  null  hypothesis  (H02)  in  favor  of  the  alternative  (Ha2). 

The  Median  test  (Table  8)  and  its  test  statistics  (Table  9)  indicated  that,  both,  the 
treatment  and  control  groups  of  MG  3  (naive  group),  achieved  significant  improvements  in  their 
scores  from  the  Day  1  Pretest  to  the  Day  1  Poshest  (in  bold).  The  same  Median  test  and  statistics 
revealed  a  significant  difference  by  experience  levels  in  performance  gains  (change  scores) 
between  the  Day  1  Poshest  and  the  Day  7  Poshest.  The  majority  of  the  naive  (MG  3)  treatment 
group  and  of  the  control  groups  of  the  minimally-experienced  (MG  2)  and  naive  (MG  3) 
participants  improved  their  scores  greater  than  the  median  improvement  of  their  respective  group 
populations  (in  bold.  Table  8). 

The  majority  of  the  highly-experienced  group  did  not  show  performance  gains  greater 
than  the  median  of  their  respective  groups.  This  is  likely  due  to  a  ceiling  effect.  That  is,  their 
preexisting  high  level  of  EP  Knowledge  competence  may  be  so  high  that  treatment  effects  are 
not  perceptible  or  obscured  (Clarion  University  of  Pennsylvania,  n.d.). 
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Table  7. 

Kruskal- Wallis  Test  statisties  for  Emergeney  Proeedures  Knowledge  test  etiange  seores  by 
treatment  and  eontrol  membership  groups  with  respeet  to  experienee. 


Day  1  Pre-/Posttest 

Day  1/Day  7  Posttests 

Treatment 

Control 

Treatment 

Control 

Chi-Square 

24.442 

22.905 

17.012 

6.155 

df 

2 

2 

2 

2 

Asymptotic 

.000** 

.000** 

.000** 

.046* 

Significance 

Note:  Grouping  Variable:  Membership  Group;  *  p  <  .05;  **p  <  .01. 


Table  8. 

Median  Test  for  Emergeney  Proeedures  Knowledge  test  ehange  seores  by  treatment  and  eontrol 
membership  groups  with  respect  to  experience. 


Treatment  Groups 

Control  Groups 

MG  1 

MG  2 

MG  3 

MG  1 

MG  2 

MG  3 

Day  1 

Pre -/Posttest 

>  Median 

3 

3 

14 

3 

4 

14 

<=  Median 

12 

14 

2 

12 

11 

1 

Day  1/Day  7 
Posttests 

>  Median 

4 

4 

13 

2 

9 

9 

<=  Median 

11 

13 

3 

13 

6 

6 

Note;  MG  =  membership  group 
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Table  9. 

Median  Test  statisties  for  Emergeney  Proeedures  Knowledge  test  ehange  seores  by  treatment  and 
eontrol  membership  groups  with  respeet  to  experience. 


Day  1  Pre-/Posttest 

Day  1/Day  7  Posttests 

Treatment 

Control 

Treatment 

Control 

n 

48 

45 

48 

45 

Median 

8.00 

12.00 

4.00 

4.00 

Chi-Square 

20.761 

19.821 

13.746 

8.82 

df 

2 

2 

2 

2 

Asymptotic 

.000** 

.000** 

.001** 

.012* 

Significance 

Note:  Grouping  Variable:  Membership  Group;  *  p  <  .05;  **p  <  .01. 


Hypotheses  H03  and  Ha3 

The  Kruskal- Wallis  and  Median  tests  were  again  employed  to  address  Hypotheses  H03 
and  Ha3  due  to  the  data’s  non-normal  distribution.  The  research  question  inquired  whether 
individual  learning  style  preferences  (visual,  aural,  read/write,  kinesthetic,  or  multimodal)  had  an 
effect  on  the  recall  performance  of  individual  membership  groups  taught  using  either  the 
traditional  method  or  the  IPS  method.  Figure  8  summarizes  the  number  of  participants  classified 
through  the  VARK  Questionnaire  (Appendix  C)  as  to  their  learning  style  preference  per 
membership  group.  The  great  majority  was  classified  as  multimodal  learners,  preferring  a 
variety  of  presentation  modalities.  Note  the  absence  of  any  participants  categorized  as  strictly 
visual  learners. 
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o 

o 


□  MG1  IPS 

I  I  MG1  Traditional 

□  MG2  IPS 

I  I  MG2  Traditional 
■  MGS  IPS 
H  MGS  Traditional 


Learning  Style  Preference 


Figure  8.  Summary  of  learning  style  preference  by  membership  group.  (MG  =  membership 
group;  MM  =  multimodal  learner,  A  =  aural  learner,  R  =  read/write  learner,  K  =  kinesthetic 
learner) 


The  results  of  the  Kruskal- Wallis  and  Median  tests,  presented  in  Tables  10  through  14, 
show  that  individual  learning  style  preferences  had  no  statistically  significant  effect  on 
performance  improvements  from  one  EP  Knowledge  test  to  another  for  any  membership  group, 
whether  in  a  treatment  or  control  group. 

In  addition,  in  order  to  firmly  establish  the  absence  of  any  relationships  between 
performance  gains  and  learning  styles  preferences,  nonparametric  correlations  (Spearman’s  rho) 
were  performed.  Table  15  contains  the  results  that  revealed  no  significant  relationships. 
Considering  the  outcome  of  these  tests,  the  null  hypothesis  (H03)  was  not  rejected. 
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Table  10. 

Day  1  Pre-/Posttest:  Kruskal- Wallis  test  statistics  for  Emergency  Procedures  Knowledge  test 
change  scores  with  respect  to  learning  style  preference. 


Day  1  Pre-/Posttest 
Treatment  Groups 

Day  1  Pre-/Posttest 

Control  Groups 

MG  1 

MG  2 

MG  3 

MG  1 

MG  2 

MG  3 

Chi-Square 

4.990 

2.865 

1.635 

2.919 

.219 

2.869 

df 

2 

2 

1 

2 

2 

3 

Asymptotic 

Significance 

.083 

.239 

.201 

.232 

.896 

.412 

Note:  Grouping  Variable:  Learning  Style  Preference;  MG  =  membership  group;  *  p  <  .05; 

**p  <  .01. 


Table  11. 

Day  1/Day  7  Posttests:  Kruskal- Wallis  test  statistics  for  Emergency  Procedures  Knowledge  test 
change  scores  with  respect  to  learning  style  preference. 


Day  1/Day  7  Posttests 
Treatment  Groups 

Day  1/Day  7  Posttests 
Control  Groups 

MG  1 

MG  2 

MG  3 

MG  1 

MG  2 

MG  3 

Chi-Square 

2.906 

.260 

.026 

2.823 

2.608 

6.001 

df 

2 

2 

1 

2 

2 

3 

Asymptotic 

Significance 

.234 

.878 

.873 

.244 

.271 

.112 

Note:  Grouping  Variable:  Learning  Style  Preference;  MG  =  membership  group;  *  p  <  .05; 

**p  <  .01. 
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Table  12, 

Median  test  for  Emergeney  Proeedures  Knowledge  test  ehange  seores  by  treatment  and  eontrol 
membership  groups  with  respeet  to  learning  style  preferenee. 


Note:  MG  =  membership  group;  MM  =  multimodal  learner;  A  =  aural  learner;  R  = 
read/write  learner;  K  =  kinesthetic  learner. 
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Table  13. 

Day  1  Pre -/Posttest:  Median  test  statisties  for  Emergeney  Proeedures  Knowledge  test  etiange 
seores  by  treatment  and  eontrol  membership  groups  with  respeet  to  learning  style  preferenee. 


Day  1  Pre-/Posttest 
Treatment  Groups 

Day  1  Pre-/Posttest 

Control  Groups 

MG  1 

MG  2 

MG  3 

MG  1 

MG  2 

MG  3 

n 

15 

17 

16 

15 

15 

15 

Median 

.00 

.00 

22.00 

8.00 

12.00 

44.00 

Chi-Square 

2.963 

1.231 

2.286 

.185 

3.223 

1.553 

df 

2 

2 

1 

2 

2 

3 

Asymptotic 

Significance 

.227 

.540 

.131 

.912 

.200 

.670 

Note:  Grouping  Variable:  Learning  Style  Preference;  MG  =  membership  group;  *  p  <  .05; 

**p  <  .01. 


Table  14. 

Day  1/Day  7  Posttests:  Median  test  statisties  for  Emergeney  Proeedure  Knowledge  test  ehange 
seores  by  treatment  and  eontrol  membership  groups  with  respeet  to  learning  style  preferenee. 


Day  1/Day  7  Posttests 
Treatment  Groups 

Day  1/Day  7  Posttests 
Control  Groups 

MG  1 

MG  2 

MG  3 

MG  1 

MG  2 

MG  3 

n 

15 

17 

16 

15 

15 

15 

Median 

.00 

-4.00 

18.00 

.00 

12.00 

12.00 

Chi-Square 

3.000 

1.231 

.000 

2.963 

1.616 

3.636 

df 

2 

2 

1 

2 

2 

3 

Asymptotic 

Significance 

.223 

.540 

1.000 

.227 

.446 

.304 

Note:  Grouping  Variable:  Learning  Style  Preference;  MG  =  membership  group;  *  p  <  .05; 

**p  <  .01. 
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Table  15. 

Correlations  between  Emergeney  Proeedures  Knowledge  test  ehange  seores  by  treatment  and 
eontrol  membership  groups  with  respeet  to  learning  style  preferenee.  (Correlation  eoeflioients  / 
j!7-values) 


Groups 

n 

Day  1  Pretest 
/  Day  1  Posttest 
Change  Scores 

Day  1/  Day  7 
Posttest 
Change  Scores 

All  MGs 

93 

-.157/p  =  .132 

.013 /p  =  . 904 

All  MGs  Treatment 

48 

-.127/p  =  .389 

-.094/;?  =  .526 

All  MGs  Control 

45 

-.222 //?  =  .! 42 

-.144//?  =  .345 

Learning 

MG  1  Treatment 

15 

-.101  lp  =  .l\9 

-.145/;?  =  .606 

Style 

MG  1  Control 

15 

-.456//?  =  .088 

.435  //?  =  .105 

Preferences 

MG  2  Treatment 

17 

.260 /p  =  . 3 14 

.126/p  =  .631 

MG  2  Control 

15 

.122//?  =  .664 

.273  //?  =  .324 

MG  3  Treatment 

16 

-.330/p  =  .212 

.041  /p  =  .879 

MG  3  Control 

15 

-.290 /p  =  . 294 

-.175 /p  =  . 533 

Note:  MG  =  membership  group;  *  p  <  .05;  **p  <  .01. 


Hypotheses  H04  and  Ha4 

To  test  these  hypotheses,  nonparametric  eorrelation  proeedures,  using  Spearman’s  rho, 
were  performed  to  measure  the  relationship,  if  any,  of  reeall  performanee  (as  measured  by  the 
seores  of  the  Day  1  and  Day  7  Posttests  and  the  change  scores  from  Day  1  to  Day  7)  and  of  the 
treatment  groups’  subjective  assessments  of  the  IPS  (from  positive  to  negative).  In  other  words, 
did  the  treatment  groups’  performance  and  performance  changes  have  an  influence  on  their 
opinions  of  the  IPS?  The  results  (correlation  coefficients  and  p  values)  of  the  Spearman’s  rho 
correlations  are  displayed  in  Table  16.  No  statistically  significant  correlations  were  discovered. 
The  findings  support  the  null  hypothesis  (H04)  and  reject  the  alternative  (Ha4)- 
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Table  16. 

Correlations  between  subjeetive  assessment  eategories  and  Emergeney  Proeedure  Knowledge 
test  seores  and  ehange  seores.  (Correlation  eoeffieients  /  j!?-values) 


Day  1  EP 
Knowledge 
Posttest 

Day  7  EP 
Knowledge 
Posttest 

Day  1/  Day  7 
Posttest  Change 
Scores 

Subjective 

MG  1  (n=  15) 

-.054//?  =  .848 

-.256 !  p  =  .356 

-.390//?  =  .150 

Assessment 

^HHI 

-.043//?  =  .870 

.100//?  =  .704 

.186//?  =  .475 

Categories 

MG3  (n=  16) 

-.158/^3  =  .559 

-.221  /p  =  . 412 

-.063  /p  =  . 817 

Note:  MG  =  membership  group;  *  p  <  .05;  **p  <  .01. 


Hypotheses  Hqs  and  Has 

In  order  to  address  Hqs  and  Has  and  explore  the  effect  of  learning  style  preferences  of  the 
treatment  groups  on  their  assessment  of  the  IPS,  a  crosstabulation  of  the  data  was  conducted.  A 
crosstabulation  displays  the  number  of  cases  in  each  category  defined  by  two  or  more  categorical 
variables.  A  Pearson’s  chi-square  test  measures  whether  the  row  and  column  variables  in  a 
crosstabulation  are  independent  of  each  other  (SPSS  12.0).  Recall  that  the  learning  style 
preferences  can  be  visual,  aural,  read/write,  kinesthetic,  or  multimodal.  The  subjective 
assessment  categories  are  positive  assessment,  positive  to  neutral  assessment,  neutral  assessment, 
neutral  to  negative  assessment,  and  negative  assessment.  Table  17  contains  the  crosstabulations 
for  each  treatment  membership  group  individually  and  the  treatment  membership  groups  in 
aggregate.  Note  that  graphical  representations  (Figures  0-4  through  0-7)  of  the  crosstabulations 
are  available  in  Appendix  O.  Table  18  follows  the  crosstabulation  table  with  each 
crosstabulation’s  associated  Pearson  chi-square  test.  The  results  provided  evidence  that  the  two 
variables  were  independent  of  each  other  and  hence,  learning  style  preferences  had  no 
statistically  significant  effect  on  the  subjective  assessment  of  the  IPS  regardless  of  experience 
level  (membership  group),  whether  examined  individually  or  in  aggregate.  These  findings 
support  that  the  null  hypothesis  (H05)  is  true,  and  therefore,  it  was  not  rejected. 
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Table  17. 

Learning  style  preferenee  and  subjeetive  assessment  eategory  erosstabulations. 


Subjective  Assessment  Categories 

Positive 

Neutral 

Positive 

to  Neutral 

Neutral 

to  Negative 

Negative 

Total 

1 

7 

0 

1 

0 

9 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

0 

0 

0 

3 

0 

2 

0 

1 

0 

3 

2 

12 

0 

0 

0 

14 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

2 

11 

3 

0 

0 

0 

14 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

0 

0 

0 

2 

14 

22 

0 

1 

0 

37 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

3 

0 

0 

0 

4 

2 

4 

0 

1 

0 

7 

Note:  MG  =  membership  group;  MM  =  multimodal  learner;  A  =  aural  learner; 


R  =  read/write  learner;  K  =  kinesthetic  learner. 
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Table  18. 

Pearson’s  ehi-square  tests  resulting  from  learning  style  preferenee  and  Subjeetive  Assessment 
Category  erosstabulations. 


df 

Asymptotic 

Significance 

(2-sided) 

MG  I 

2.222 

4 

.695 

MG  2 

4.694 

2 

.096 

MGS 

.762 

1 

.383 

All  MGs 

2.512 

4 

.642 

Note:  MG  =  membership  group;  *  p  <  .05;  **p  <  .01. 


Hypotheses  Hoe  and  Ha6 

As  in  the  previous  question,  both  variables  are  categorieal  variables.  As  such,  a 
crosstabulation  (Table  19)  was  conducted  from  which  a  Pearson’s  Chi-square  test  was  used  to 
determine  whether  experience  levels,  characterized  by  the  membership  groups,  were  independent 
of  the  subjective  assessment  of  the  IPS  based  on  the  categories  derived  from  the  Subjective 
Assessment  Survey  (Appendix  D)  scores.  The  results  of  the  Pearson’s  chi-square  test  showed 
that  the  two  variables  were  not  independent  [x  (4,  N=  48)  =  20.651,  p  <  .001].  Cramer’s  V  and 
the  contingency  coefficient,  nominal  symmetric  measures  of  strength  and  significance  of  the 
relationship,  indicated  that  there  was  a  moderately  strong,  highly  significant  relationship  between 
the  two  crosstabulation  variables  (.464, <  .001  and  .549,  p  <  .001,  respectively). 

An  examination  of  Tables  19  and  20  clearly  shows  that  the  degree  of  positive  assessment 
is  inversely  related  to  the  extent  of  aviation  experience.  As  such,  the  null  hypothesis  (Hoe)  is 
rejected  in  favor  of  the  alternative  (Ha6)  which  submits  that  experience  does  have  an  effect  on 
the  subjective  assessment  of  the  IPS. 
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Table  19. 

Membership  group  and  subjeetive  assessment  eategory  erosstabulation. 


Subjective  Assessment  Categories 

Total 

Positive 

Positive 
to  Neutral 

Neutral 

Neutral 
to  Negative 

Negative 

MG  1 

1 

12 

0 

2 

0 

15 

MG  2 

4 

13 

0 

0 

0 

17 

MG  3 

12 

4 

0 

0 

0 

16 

Note:  MG  =  membership  group. 


Table  20, 

Subjeetive  assessment  raw  seores  deseriptive  statistics. 


Membership 

Group 

Mean* 

N 

Std. 

Deviation 

Min 

Max 

MG  1 

35.13 

15 

5.680 

23 

45 

MG  2 

28.76 

17 

6.638 

16 

40 

MG  3 

23.69 

16 

6.681 

14 

36 

Note:  *  The  lower  the  score  mean,  the  more  positive 


y  inclined  the  assessment; 


MG  =  membership  group;  min  =  minimum;  max  =  maximum. 


Hypotheses  Ho?  and  Ha? 

The  hypotheses  of  whether  the  IPS  was  assessed  positively  by  the  majority  of  those  in  the 
treatment  groups  as  a  useful  pictorial  mnemonic  for  remembering  aviation  emergency 
procedures  was  answered  through  the  following  descriptive  statistics.  Even  though  significant 
differences  were  found  in  IPS  assessments  depending  on  aviator  experience  (Hao),  the 
overwhelming  majority,  nearly  96%  of  the  treatment  population,  rated  the  IPS  in  a  positive 
manner  (above  neutral  and  as  classified  using  the  scoring  procedures  outlined  in  Appendix  E). 
Table  21  contains  the  overall  frequency  of  assessment  ratings,  while  Eigure  9  presents  the 
assessment  ratings  by  sample  population  percentages.  These  findings  provide  that  the  alternative 
hypothesis  (Ha?)  is  accepted  as  true  and,  thus,  refute  the  null  hypothesis  (Ho?). 
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Table  21, 

Frequency  of  overall  IPS  ratings. 


Assessment 

Frequency 

Positive 

17 

Positive  to  Neutral 

29 

Neutral 

0 

Neutral  to  Negative 

2 

Negative 

0 

Total 

48 

Figure  9.  Sample  population  percentages  of  overall  IPS  ratings  by  category. 
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To  achieve  a  more  substantive  and  insightful  understanding  of  the  IPS  assessments,  an 
examination  of  the  unproeessed  responses  to  each  question  is  useful.  Table  22  provides  the 
frequeney  of  responses  per  abbreviated  survey  statement.  A  review  of  the  responses 
demonstrates  the  IPS  was,  by  and  large,  favorably  appraised. 

Table  22. 

Subjeetive  assessment  responses  per  survey  statement. 


Subjective  Assessment  Categories 

Abbreviated  Assessment  Statements 

Strongly 

Disagree 

Disagree 

Neutral 

Agree 

Strongly 

Agree 

1 .  IPS  not  a  useful  tool. 

14 

21 

2 

11 

0 

2.  Prefer  my  own  system  over  IPS. 

5 

15 

26 

2 

0 

3. 1  would  not  use  IPS  in  the  future. 

9 

26 

6 

5 

2 

4.  IPS  adds  to  the  edueational 
burden. 

11 

29 

5 

3 

0 

5.  IPS  symbols  not  intuitive. 

17 

23 

3 

5 

0 

6.  IPS  too  complex  for  new  student. 

18 

29 

1 

0 

0 

7.  IPS  has  not  improved  EP  reeall. 

8 

23 

12 

5 

0 

8.  Don’t  teach  IPS  to  future  students. 

18 

22 

8 

0 

0 

9.  IPS  did  not  assist  me. 

7 

24 

12 

5 

0 

10.  IPS  is  a  waste  of  valuable  time. 

10 

31 

6 

1 

0 

1 1 .  Would  not  reeommend  IPS  to 
the  Army. 

14 

27 

6 

1 

0 

12.  IPS  may  eonfuse  pilot  majority. 

11 

32 

4 

1 

0 

13.  IPS  should  not  be  used  in  EP 
cheeklist  instead  of  text. 

10 

14 

16 

6 

1 

14.  IPS  could  cause  confusion  during 
actual  emergency  situation. 

11 

23 

10 

4 

0 
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Finally,  the  Subjective  Assessment  Survey  (Appendix  D)  solicited  from  participants 
additional  comments  regarding  the  IPS,  the  intuitiveness  of  the  symbols,  and  recommendations 
for  future  applications.  Thirty-two  of  the  48  treatment  participants  provided  additional 
comments  which  are  located  in  Appendix  P.  A  review  of  the  remarks  indicated  that  50%  (16) 
were  laudatory  in  nature;  25%  (8)  recommended  specific  symbol  changes;  15.6%  (5)  were 
laudatory  of  the  IPS,  yet  negatively  critical  of  some  of  the  symbols;  and  9.4%  (3)  were 
negatively  critical  of  the  IPS  as  a  mnemonic  system,  citing  the  possibility  of  causing  confusion 
and/or  misinterpretation. 


Hypotheses  Hog  and  Has 

The  testing  of  Hypotheses  Hog  and  Has  sought  to  resolve  whether  the  symbols  that  make 
up  the  IPS  were  easily  recognized  (intuitive)  and,  thus,  remembered  (having  memorable 
qualities).  To  answer  the  question,  paired  samples  t  tests  were  used  to  compare  the  means  of  the 
Symbol  Recognition  Tests  (Appendix  F)  administered  on  Days  1  and  7.  Table  23  presents 
descriptive  statistics  for  the  Day  1  and  Day  7  Symbol  Recognition  Tests  for  each  membership 
group  and  all  membership  groups  combined. 

Recall  that  the  Day  1  test  was  administered  immediately  following  the  participants’ 
introduction  to  the  IPS  symbols.  Taking  into  account  the  very  limited  exposure  to  the  symbols 
(less  than  30  minutes)  prior  to  the  test,  each  group  scored,  on  average,  at  least  75%  correctly 
(Figure  10),  with  22.9%  of  the  sample  population  achieving  a  perfect  score. 

Table  23. 

Symbol  Recognition  test  score  descriptive  statistics. 


Membership 

Group 

n 

Mean 

Std. 

Deviation 

Std.  Error 
Mean 

Min 

Max 

Day  1 

Symbol 

Recognition 

Test 

MG  1 

15 

85.00 

16.S66 

4.226 

45 

100 

MG  2 

17 

92.S5 

8.S14 

2.016 

75 

100 

MGS 

16 

75.S1 

1S.S50 

S.SS8 

40 

95 

All  MGs 

48 

84.S8 

14.5S6 

2.098 

40 

100 

Day  7 

Symbol 

Recognition 

Test 

MG  1 

15 

85.00 

11.802 

4.226 

60 

100 

MG  2 

17 

81.18 

15.765 

S.824 

45 

100 

MGS 

16 

85.00 

12.S8S 

S.096 

55 

100 

All  MGs 

48 

8S.65 

1S.S58 

1.928 

45 

100 

Note:  MG  =  membership  group;  min  =  minimum;  max  =  maximum. 
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Symbol 

r~l  Recognition 
Test  1 

Symbol 

r~l  Recognition 
Test  2 


MG1  IPS  MG2  IPS  MGS  IPS 

Membership  Group 

Figure  10.  Symbol  Recognition  Test  score  means  by  membership  group.  (MG  =  membership 
group) 

When  tested  as  a  collective  body,  the  participants’  symbol  recognition  scores  did  not 
differ  significantly  [t(47)  =  .290, />  =  .773]  from  one  week  {M=  84.38)  to  the  next  (M=  83.65). 
Similar  results  were  achieved  when  comparisons  of  the  MG  1  (highly-experienced)  Day  1  and 
Day  7  test  scores  (M=  85.00  andM=  85.00,  respectively)  were  conducted.  Obviously,  since  the 
values  were  the  same,  the  paired  samples  tests  for  MG  1  showed  no  statistically  significant 
performance  differences  between  the  two  tests  [t(14)  =  .000, =  1.000]. 

In  contrast,  the  minimally-experienced  (MG  2)  and  naive  (MG  3)  groups  showed 
significant  differences  in  their  test  performances  from  Day  1  to  Day  7,  albeit  in  different 
directions.  The  paired  samples  test  for  MG  2  showed  a  statistically  significant  decline  in 
performance  from  one  week  (M=  92.35)  to  the  next  (M=  81.18)  [t(16)  =  3.297, =  .005].  MG 
3,  on  the  other  hand,  demonstrated  a  statistically  significant  [t(15)  =  -3.183, />  =  .006] 
improvement  in  their  test  performance  from  Day  1  to  Day  7  (fromM=  75.31  to  M=  85.00). 

It  is  important  to  realize  that  the  minimally-experienced  group  (MG  2)  showed  a  decline 
from  an  average  score  on  the  first  test  that  was  seven  points  higher  than  the  highly-experienced 
group  (MG  1)  (92%  vs.  85%,  respectively)  to  a  score  one  week  later  that  was  just  four  points 
lower  the  highly-experienced  group  (81%  vs.  85%).  Also  note  that  the  naive  group  (MG  3) 
achieved  the  same  level  of  symbol  recognition  on  Day  7  as  the  highly  experienced  group  (both 
with  85%  scores).  The  importance  of  these  observations  is  that  on  both  tests,  the  average  of  the 
least-experienced  participants  (MG  2  and  MG  3)  scored  nearly  the  same  or  better  than  the 
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highly-experienced  group  (MG  1),  who  have  a  minimum  of  four  years  experience  and  familiarity 
memorizing  the  emergency  procedures  by  text.  The  performance  of  the  less  experienced  groups, 
especially  the  naive  group,  in  closely  matching  the  highly-experienced  group  demonstrates  the 
ability  of  the  symbols  to  bring  back  uncommon  text  for  which  there  was  minimal  or  no 
familiarization. 

For  the  treatment  group  as  a  whole  and  for  the  highly-experienced  group  (MG  1) 
individually,  support  was  for  the  null  hypothesis  (Hog)  as  being  true  and  indications  are  that 
recall  of  the  symbols  was  maintained  without  any  significant  change.  Conversely,  the  significant 
differences  in  performance  for  the  minimally-experienced  (MG  2)  and  naive  (MG  3)  groups  lend 
support  for  the  alternative  hypothesis  (Has).  Notwithstanding  the  performance  decrement  of  MG 
2  to  levels  nearly  the  same  as  the  other  groups,  the  maintenance  or  improvement  in  IPS  symbol 
recall  by  the  majority  of  participants  provides  encouraging  evidence  as  to  the  intuitive  nature  and 
memorable  qualities  of  the  IPS. 


Hypotheses  Hog  and  Ha9 

Hypotheses  Hog  and  Ha9  were  tested  to  uncover  any  possible  associations  between  the 
time  from  Day  I  to  Day  7  committed  to  study  and  the  performance  gains  on  the  EP  Knowledge 
and  Symbol  Recognition  posttests.  The  Study  Experience  Report  (Appendix  J)  was  used  to 
collect  data  from  which  descriptive  statistics  were  derived  and  possible  relationships  between 
committed  study  time  and  test  results  were  explored.  Boxplots  of  individual  group  data  were 
examined  to  identify  gross  outliers.  The  boxplot  (Eigure  1 1)  of  the  data  of  the  minimally- 
experienced  control  group  (MG  2)  revealed  the  presence  of  three  gross  outliers  greater  than  five 
standard  deviations  from  the  mean.  It  is  possible  that  the  presence  of  these  outliers  may  have 
resulted  from  the  misinterpretation  of  the  data  collection  instructions  by  these  individuals.  The 
three  outliers  were  normalized  (replaced  with  the  mean  of  the  remaining  minimally-experienced 
group  members)  prior  to  analysis.  In  addition  to  time  committed  to  the  overall  study  of 
emergency  procedures,  members  of  the  treatment  groups  were  also  requested  to  provide  an 
estimate  of  their  time  using  the  IPS  while  studying. 
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Figure  11.  Boxplot  of  minimally-experienced  control  group’s  study  time.  (EP  =  emergency 
procedure;  MG  =  membership  group) 


An  examination  of  the  descriptive  statistics  (Table  24)  shows  that  in  almost  every  case 
(the  naive  treatment  group  using  IPS  being  the  only  exception),  the  control  groups  reported  a 
greater  daily  average  time  studying  than  the  treatment  groups.  The  results  of  Kruskal-Wallis  and 
Median  tests  (Tables  25  through  27)  indicated  no  significant  differences  in  study  time  by 
treatment  group  members  when  using  the  IPS.  On  the  other  hand,  a  significant  group  difference 
was  detected  in  the  study  time  when  studying  without  the  IPS.  Naturally,  this  population 
included  treatment  and  control  group  members.  The  significance  was  due  to  the  fact  that  all  15 
members  of  the  minimally-experienced  control  group  (in  bold.  Table  26)  studied  significantly 
more  than  the  median  time  of  the  overall  group  population.  There  seems  no  reasonable 
explanation  for  this  randomly  assigned  group’s  extraordinary  penchant  for  studying  other  than 
chance. 
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Table  24, 

Study  time  deseriptive  statisties  (hours  per  day). 


Membership 

Treatment/ 

Std. 

Group 

Control 

Mean 

N 

Deviation 

Min 

Max 

Treatment 

.3667 

15 

.56779 

.00 

2.00 

MG  1 

(.2867) 

(.54231) 

(.00) 

(2.00) 

Estimate  of  time 

Control 

.4500 

15 

.60622 

.00 

2.00 

studying  EPs 
without  IPS 

MG  2 

Treatment 

.3353 

17 

.53379 

.00 

2.00 

(Estimate  of  time 

(.2765) 

(.54946) 

(.00) 

(2.00) 

Control* 

.9967 

15 

.51077 

.25 

2.00 

using  IPS  to  study 

EPs) 

MG  3 

Treatment 

.2438 

16 

.22721 

.00 

1.00 

(.2906) 

(.30233) 

(.00) 

(1.00) 

Control 

.2900 

15 

.29000 

.10 

1.00 

Note:  *Outliers  normalized;  MG  =  membership  group; 


:P  =  emergency  procedures;  IPS 


Intuitive  Pictorial  System;  min  =  minimum;  max  =  maximum. 


Table  25. 

Kruskal- Wallis  test  statistics  for  reported  emergency  procedures  study  time. 


EP  Study  Time  Without  IPS 

EP  Study  Time  Using  IPS 

Treatment  &  Control  Groups 

Treatment  Groups  only 

Chi-Square 

23.426 

4.790 

df 

5 

2 

Asymptotic 

Significance 

.000** 

.091 

Note:  Grouping  Variable:  Membership  Group;  *  p  <  .05;  **p  <  .01;  EP  =  emergency 
procedures;  IPS  =  Intuitive  Pictorial  System. 
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Table  26. 

Median  test  for  reported  emergeney  proeedures  study  time. 


Treatment  Groups 

Control  Groups 

MG  I 

MG  2 

MGS 

MG  1 

MG  2 

MGS 

EP  Study 

Time  without 

using  IPS 

>  Median 

6 

4 

6 

7 

15 

6 

<=  Median 

9 

13 

10 

8 

0 

9 

EP  Study 

Time  using 

IPS 

>  Median 

6 

4 

10 

* 

* 

* 

<=  Median 

9 

II 

6 

* 

Note:  *  Control  Groups  were  not  exposed  to  IPS  (Intuitive  Pietorial  System); 
MG  =  membership  group;  EP  =  emergency  procedures. 


Table  27. 

Median  test  statisties  for  reported  emergeney  proeedures  study  time. 


EP  Study  Time  without  IPS 

EP  Study  Time  using  IPS 

Treatment  &  Control  Groups 

Treatment  Group  only 

n 

93 

48 

Median 

.2000 

.1000 

Chi-Square 

21.826 

2.756 

df 

5 

2 

Asymptotic 

Significance 

.001** 

.252 

Note:  Grouping  Variable:  Membership  Group;  *  p  <  .05;  **p  <  .01;  EP  =  emergeney 
proeedures;  IPS  =  Intuitive  Pictorial  System. 
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The  results  of  a  series  of  Spearman’s  rho  correlations  revealed  that  only  two  statistically 
significant  relationships  existed  between  the  reported  study  times  and  participant  performance. 
The  significant  correlations  were  found  relevant  to  the  highly-experienced  (MG  1)  treatment 
group  only.  The  tests  found  that  there  were  significant  positive  correlations  between  participant 
study  time  using  the  IPS  and  improvements  (as  evidenced  by  change  scores)  in  both  the  EP 
Knowledge  and  Symbol  Recognition  tests  [^^(13)  =  .523, p=  .045  and  r^lS)  =  529, p=  .043, 
respectively].  Stated  clearly,  as  the  members  of  the  MG  1  treatment  group’s  one-week  study 
time  increased,  so  did  the  improvements  in  their  test  scores. 

Hence,  with  regard  to  the  highly-experienced  MG  1  group,  there  was  a  significant 
association  between  committed  study  time  during  the  week  and  test  performance  gains.  These 
findings  refute  the  null  hypothesis  (H09)  and  support  the  alternative  (Ha9).  Even  though  the 
minimally-experienced  (MG  2)  group  studied  significantly  more  than  the  other  groups,  there  was 
no  significant  correlation  between  study  time  and  performance  gains.  Similarly,  there  was  no 
correlation  revealed  regarding  the  naiVe  (MG  3)  group  either.  Thus,  for  these  two  groups,  the 
null  hypothesis  was  true. 


Discussion  and  Implications  of  the  Research 

The  primary  purpose  of  this  study  was  to  determine  the  utility  and  merit  of  using  the  IPS 
as  a  mnemonic  strategy  to  facilitate  the  learning  and  accurate  recall  of  aviation  emergency 
procedures.  The  research  methodology  used  data  from  three  distinct  membership  groups  based 
on  their  levels  of  aviation  experience.  As  such,  each  group  provides  unique  insights  into  the 
assessment,  utility,  and  merit  of  the  IPS.  The  following  discussion  attempts  to  highlight  these 
sometimes  subtle  insights. 


Research  Question  1 

The  results  of  the  statistical  tests  conducted  to  answer  RQl  determined  that  the  IPS  did 
not  improve  retention  and  recall  in  the  overall  sample  population  over  that  of  the  traditional 
method  from  Day  1  EP  Knowledge  Pretest  to  Day  1  Posttest.  In  light  of  previous  research 
extolling  the  ease  and  capacity  for  remembering  pictorial  representations  (Haber,  1970; 

Standing,  Conezio,  and  Haber,  1970;  Potter,  1976;  Simons,  1996;  Hollingworth,  Williams,  and 
Henderson,  2001;  Hollingworth  and  Henderson,  2002),  this  finding  was  unexpected.  Moreover, 
the  control  group  of  the  total  sample  population  achieved  a  statistically  significant  improvement 
in  their  scores.  This  finding,  taken  in  isolation,  would  indicate  a  clear  superiority  of  the 
traditional  method  over  the  IPS  method.  However,  the  comparative  results  (discussed  below)  of 
the  Day  1  to  Day  7  Posttests  are  inconsistent  with  the  above  observation  and,  thus,  do  not 
support  this  preliminary  conclusion.  The  initial  large  disparity  between  group  performances  was 
initially  puzzling;  however,  a  plausible  explanation  for  the  disparity  may  be  the  difference  in  the 
amount  of  information  presented  to  each  group. 

It  is  important  to  note  that  the  Day  I  EP  Knowledge  Posttest  was  administered 
immediately  following  the  EP  Instructional  PowerPoint  Presentation  (Appendices  G  or  H,  as 
appropriate)  and  that  two-thirds  of  the  total  sample  population  (MG  1  and  MG  2)  had  previous 
memorization  experience  with  the  EPs.  Thus,  the  EP  Instructional  PowerPoint  Presentation 
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served  the  majority  of  the  control  group  as  a  review  and  reinforcement  of  existing/established 
memory  strategies.  In  contrast,  those  in  the  treatment  group  were  presented  with  the  same  EPs, 
plus,  were  directed  to  employ  a  new  memorization  strategy  as  their  memorization  technique. 

The  30-minutes  to  one-hour  interval  between  Day  1  EP  Knowledge  Pretest  and  Posttest  may  not 
have  been  sufficient  time  to  assimilate  the  amount  of  new  information  presented. 

The  plausibility  of  the  explanation  is  supported  when  the  Day  1  and  Day  7  Posttests  are 
compared  and  the  results  are  inconsistent  with  the  findings  above.  In  this  comparison,  the 
average  score  improvements  made  by  the  sample  population’s  treatment  and  control  groups  did 
not  differ  significantly  from  each  other.  With  regard  to  the  research  question,  the  IPS  still  did 
not  improve  retention  and  recall  over  that  of  the  traditional  method  in  the  overall  population; 
however,  neither  method  was  significantly  superior  to  the  other.  What  is  more,  participants 
receiving  either  method  of  instruction  made  statistically  significant  improvements,  on  average,  in 
their  EP  Knowledge  posttest  performances  from  Day  1  to  Day  7.  All  things  considered,  neither 
instructional  method  demonstrated  conclusive  superiority  over  the  other. 

Research  Question  2 

In  determining  if  there  was  a  difference  in  recall  performance  between  the  IPS  and 
traditional  methods  with  regard  to  experience  levels,  the  statistics  revealed  that  significant 
differences  in  performance  were  indeed  influenced  by  experience  levels  with  the  only  difference 
appearing  in  the  minimally-experienced  population  (MG  2). 

Not  unexpectedly,  neither  training  method  had  an  affect  on  the  performance  of  the 
highly-experienced  group  (MG  1)  who  showed  no  significant  improvements  in  any  of  their  tests. 
As  noted  earlier,  this  may  be  due  to  a  ceiling  effect.  In  other  words,  participants  of  this  group 
started  with  relatively  high  Pretest  scores  which  then  remained  high  on  their  two  posttests. 

As  in  the  case  with  the  highly-experienced  group,  the  training  method  made  no 
significant  difference  in  the  performance  improvements  of  the  naive  group  (MG  3)  as  both 
treatment  and  control  groups  made  significant  gains  in  their  performances.  Of  interest  is  that 
from  the  Day  1  Pretest  to  Day  1  Posttest,  the  control  group  made  the  greatest  average 
improvement  (41  vs.  22  points).  But,  from  the  Day  1  to  Day  7  Posttest,  it  was  the  treatment 
group  who  made  the  greatest  average  improvement  (21  vs.  10  points).  Again,  as  discussed  with 
reference  to  RQl,  the  difference  between  Day  1  and  Day  7  Posttest  performances  of  the 
treatment  and  control  groups  in  this  naive  population  may  be  due  to  the  difference  in  the  amount 
of  new  information  presented  to  each  group  on  Day  1 . 

The  significant  difference  between  instructional  methods  referred  to  above  was  detected 
in  the  minimally-experienced  group  (MG  2).  The  Median  test  for  this  group  illustrates  no 
practical  differences  between  treatment  and  control  groups  for  the  performance  improvements 
from  Day  1  Pretest  to  Day  1  Posttest,  but  does  show  differences  in  group  performances  from  Day 
1  to  Day  7.  Eurthermore,  it  was  the  control  group  that  performed  better  (an  1 1  point  average 
improvement),  while  the  treatment  group’s  average  score  decreased  by  3  points.  These  findings 
should  point  conclusively  to  the  superiority  of  the  traditional  method  over  the  IPS  with  regard  to 
minimally-experienced  aviators.  However,  the  findings  are  confounded  by  the  potential  effects 


50 


of  the  disproportionate  study  time  reported  by  the  two  groups  (treatment  and  eontrol)  discussed 
later  in  Research  Question  9.  Because  the  sample  population  was  actively  engaged  in  some 
capacity  in  the  U.S.  Army’s  flight  school  program  as  instructors,  active  students,  or  inactive 
students,  and  not  subjected  to  a  completely  controlled  laboratory  environment,  study  time  could 
not  be  regulated  or  controlled.  Consequently,  members  of  the  minimally-experienced  control 
group  (MG  2)  reportedly  studied  on  average  three  times  as  long  as  the  treatment  group  (.996  vs. 
.335  hours  per  day;  Table  22),  the  largest  significant  difference  within  a  membership  group. 
Therefore,  as  with  RQl,  the  findings  do  not  show  the  clear  superiority  of  either  method. 

Research  Question  3 

To  answer  this  question,  the  VARK  Questionnaire  (Appendix  C)  was  employed  to 
classify  participants’  learning  style  preferences.  It  was  used  for  pragmatic  reasons:  it  is  quick 
and  easy  to  complete  and  the  results  are  relatively  easy  to  obtain  and  interpret.  Using  the  derived 
classifications,  Kruskal- Wallis  and  Median  tests  were  employed  to  determine  if  recall 
performance  (as  measured  by  gains  in  the  EP  Knowledge  tests)  between  the  IPS  and  traditional 
methods  varied  with  respect  to  learning  style  preferences.  The  results  produced  no  statistically 
significant  differences. 

According  to  theory,  individuals  differ  in  the  manner  from  which  they  best  absorb 
information  (Harrison,  Andrews,  and  Saklofske,  2003;  Cassidy  and  Eachus,  2000;  Dunn,  1983) 
based  largely  on  the  preferred  learning  modality  (Zapalska  and  Dabb,  2002).  Specifically,  those 
with  style  preferences  should  show  the  greatest  ability  to  recall  material  presented  in  that  manner 
(Kratzig  and  Arbuthnott,  2006).  This  study’s  sample  population  was  comprised  of  very  few 
members  having  a  single  learning  style  preference  (Eigure  8)  with  the  majority  (68  of  93) 
classified  as  having  multimodal  learning  preferences.  Despite  reports  praising  the  virtues  of 
matching  learning  style  preference  with  modality  of  presentation  (Munro  and  Rice-Munro,  2004; 
Earkas,  2003;  Eohri-Posey,  2003),  this  study’s  findings  are  consistent  with  those  of  Eoo  (2004) 
and  Kratzig  and  Arbuthnott,  who  report  weak  to  no  significant  link  between  learning  style 
preference  and  objective  memory  performance.  In  any  case,  the  findings,  at  least  in  this  sample 
population,  indicate  that  neither  the  IPS  nor  the  traditional  training  method  is  prejudicial  toward 
any  single  or  multimodal  learning  style  preferences. 

Research  Question  4 

According  to  the  Instructor  Pilot  Handbook  (U.S.  Army  Aviation  Center,  2000),  the 
Principle  of  Effect  states  that  learning  is  strengthened  when  accompanied  by  a  pleasant  or 
satisfying  feeling.  Conversely,  learning  experiences  that  produce  feelings  of  frustration, 
confusion,  and  futility  weaken  learning.  The  aim  of  Research  Question  4  was  to  achieve  a  sense 
for  the  emotional  effect  of  the  IPS  on  the  treated  membership  groups.  Basically,  the  question 
asked  if  participants’  performances  and  performance  gains  had  any  correlation  to  the  manner  in 
which  they  assessed  the  IPS.  Since  no  statistically  significant  correlations  were  found  between 
recall  performance  and  the  subjective  assessment  of  the  IPS,  no  inferences  or  presumptions  of 
training  effect,  either  positive  or  negative,  can  be  made  based  upon  the  relationship  of  these  two 
variables. 
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Research  Question  5 


Research  Question  5  was  an  inquiry  into  the  possibility  that  learning  style  preference 
might  predispose  one’s  subjective  assessment  of  the  IPS.  Stated  differently,  the  goal  was  to 
determine  if  those  more  likely  to  prefer  learning  in  other  than  visual  ways  might  have  a  less 
positive  opinion  of  the  IPS  than  visually-oriented  learners.  A  crosstabulation  (Table  17)  of  the 
variables  produced  no  significant  chi-square  statistics  (Table  18)  for  any  of  the  membership 
groups  or  all  membership  groups  in  aggregate.  This  means  that  the  two  variables  (learning  style 
preference  and  subjective  assessment  categories)  were  independent  and  unrelated  and 
operationally,  that  learning  style  preference  has  no  affect  on  one’s  opinion  of  the  IPS  as  a 
mnemonic  strategy. 

Although  not  significant,  it  is  interesting  to  note  that  the  only  two  members  to  rate  the 
IPS  in  a  neutral  to  negative  manner  were  two  highly-experienced  aviators:  one,  a  kinesthetic 
learner,  and  the  other,  a  multimodal  learner  (Table  17).  A  closer  examination  of  the  composite 
data  revealed  that  the  kinesthetic  learner  performed  poorly  on  the  EP  Knowledge  pretest  and 
both  posttests  (20,  15,  and  24  points  below  the  mean,  respectively).  The  multimodal  learner,  on 
the  other  hand,  achieved  performance  scores  closely  aligned  with  group  means.  These 
observations  are  of  questionable  importance,  but  may  provide  some  additional  insight. 

Research  Question  6 

To  determine  if  experience  levels  had  an  effect  on  the  subjective  assessment  of  the  IPS,  a 
crosstabultion  of  the  two  variables  was  performed.  The  resulting  chi-square  statistic  identifies  a 
moderately  strong,  highly  significant  relationship  (p  <  .001)  between  the  two  variables.  This 
indicates  that  experience  does  have  an  affect  on  the  subjective  assessment  of  the  IPS.  Simply 
put,  the  more  experience,  the  less  favorable  the  assessment.  That  being  said  and  as  revealed 
above,  the  IPS  received  only  two  assessments  that  were  negatively  inclined,  both  by  members  of 
the  highly-experienced  group.  Even  though  all  other  assessments  were  slanted  in  a  positive 
direction,  the  significant  relationship  between  experience  and  subjective  assessment  is  evident  in 
the  crosstabulation  in  Table  19.  It  is  also  apparent  in  a  comparison  of  the  group  means  in  Table 
20.  In  the  case  of  subjective  assessment  scores,  the  lower  the  score,  the  more  positive  the 
assessment.  The  highly-experienced  group’s  mean  was  35.13,  with  the  minimally-experienced 
group  at  28.76,  and  the  naive  group  at  23.69. 

A  reasonable  explanation  for  the  significant  relationship  may  be  based  on  a  common 
resistance  towards  change.  According  to  the  expectancy  theory  (Eines,  quoting  Vroom,  2004), 
“people  consciously  choose  courses  of  action,  based  upon  perceptions,  attitudes,  and  beliefs,  as  a 
consequence  of  their  desires  to  enhance  pleasure  and  avoid  pain”  (p.  198).  Expectancy  theory 
predicts  that  resistance  will  occur  if  1)  the  individual  has  expectancies  that  the  relationship 
between  a  change  in  behavior  and  performance  is  uncertain,  2)  the  link  between  performance  and 
outcome  is  uncertain,  and  3)  the  outcomes  have  negative  value  to  the  individual  (Hope  and  Pate, 
1988). 


Experience  in  aviation  implies  learned  and  established  knowledge  and  skills.  Once 
established,  any  alterations  or  redefinitions  naturally  lead  to  some  people  feeling  uncomfortable 
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or  threatened  (Walley,  1995).  Aviators,  being  professional  people,  value  their  professionalism 
and  hard-won  skills.  Any  ehanges  that  threaten  to  make  these  skills  obsolete  are  likely  to 
eneounter  resistanee,  espeeially  with  a  less  teehnieally  eompetent  aviator  who  may  beeome 
defensive  to  preserve  a  eompetent  self-image  (Jensen,  1998).  In  addition,  unfamiliar  novel 
systems,  sueh  as  the  IPS,  provoke  barriers  to  ehange  such  as  fear  of  the  unknown,  low  trust, 
arrogant  attitudes,  and  resistant  organizational  cultures  (Kane  and  Darling,  2002).  In  light  of  the 
preceding  discussion,  the  discovery  that  experience  levels  affect  the  assessment  of  the  IPS  is 
logical  and  not  unexpected. 


Research  Question  7 

Based  on  the  descriptive  statistics  derived  from  the  Subjective  Assessment  Surveys 
(Appendix  D),  answering  whether  the  IPS  was  appraised  as  a  useful  mnemonic  for  the  intended 
purpose  was  straightforward  and  unproblematic.  As  stated  in  the  previous  chapter,  96%  of  the 
sample  population  rated  the  system  in  a  favorable  manner.  Support  for  incorporating  the  IPS 
into  flight  training  was  evidenced  by  noting  that  only  one  of  the  participants  would  not 
recommend  the  system  for  use  in  the  U.S.  Army.  No  respondent  felt  that  the  IPS  was  too 
complex  for  new  flight  students  and  none  recommended  that  it  not  be  taught  to  future  flight 
students.  These  findings  support  previous  research,  such  as  conducted  by  Haber  (1970)  and 
Cherry,  Dokey,  Reese,  and  Brigman  (2003),  which  demonstrated  peoples’  predilection  for 
pictorial  representations  and  their  usefulness  as  aids  to  memory. 

Research  Question  8 

Along  with  the  results  of  the  Subjective  Assessment  Survey,  the  findings  relating  to  this 
research  question  may  be  the  single  most  important  indication  of  the  merits  and  utility  of  the  IPS 
as  an  aid  to  memory.  The  intent  of  this  research  question  was  to  determine  if  the  IPS  symbols 
were  easily  recognized  (intuitive)  and,  thus,  had  memorable  qualities.  Since  the  first  Symbol 
Recognition  test  (Appendix  F)  was  administered  within  30  minutes  of  the  participants  ever 
seeing  the  symbols,  a  mean  score  of  84.38  for  the  sample  population  was  impressive  and 
indicative  of  the  symbols’  intuitive  nature  and  memorability.  The  fact  that  every  experience 
group  averaged  high  scores  immediately  following  their  initial  presentation  showed  the  symbols 
to  be  intuitive  regardless  of  prior  knowledge  and  familiarity  with  the  represented  terms. 

When  the  test  was  administered  one  week  later,  the  mean  score  for  the  sample  population 
had  not  changed  significantly  (to  83.65).  When  each  experience  group  was  examined  closely, 
the  highly-experienced  group’s  mean  performance  showed  no  change  (85  vs.  85).  The  naive 
group  showed  a  significant  performance  improvement  over  the  one  week  period  (from  a  mean  of 
75  to  85),  while  the  minimally-experienced  group  showed  an  average  decline  in  performance 
from  92  to  81.  Although  a  significant  fall  off,  an  81%  correct  score,  on  average,  by  the 
minimally-experienced  group  still  demonstrates  strong  and  successful  recall  of  the  symbols 
nearly  equivalent  with  that  of  the  highly-experienced  group.  The  test  results  and  comparisons 
suggest  strongly  that  the  symbols  are  indeed  intuitive  and  memorable. 

Although  performance  on  the  Symbol  Recognition  tests  is  a  good  indication  of  the 
intuitiveness  of  the  current  symbols,  written  comments  received  via  the  Subjective  Assessment 
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Survey  will  aid  in  refining  the  symbols  to  enhanee  intuitiveness  and  eliminate  any  possibilities  of 
eonfusion  or  misinterpretations. 


Researeh  Question  9 

Analysis  of  study  time  data  revealed  that  the  minimally-experieneed  (MG  2)  eontrol 
group  studied  far  more  than  the  other  groups  (greater  than  twiee  as  long)  and  that  signifieant 
correlations  existed  in  the  highly-experienced  treatment  group’s  study  time  and  performance 
gains.  Although  these  findings  were  statistically  significant  and  noteworthy,  their  importance  is 
not  clear. 

As  discussed  earlier  in  Research  Question  2,  the  significant  difference  in  the  MG  2 
control  group’s  study  time  compared  to  the  treatment  group’s  may  have  played  a  role  in  the 
results  that  showed  that,  in  that  experience  level,  the  traditional  method  produced  significantly 
better  results  over  the  IPS.  Regarding  the  significant  correlations  between  study  time  and 
performance  gains  by  members  of  the  highly-experienced  treatment  group,  all  members  of  this 
group  happened  to  be  instructor  pilots.  According  to  Estrada  and  Dumond  (2006),  instructor 
pilots  study  emergency  procedures  less  frequently  than  graduate  and  student  pilots.  It  is  likely 
that  instructor  pilots  rely  on  their  daily  instructional  activities  to  maintain  a  minimal  level  of 
recall  proficiency.  Therefore,  any  small  increase  in  the  amount  of  time  committed  to  studying 
emergency  procedures  would  naturally  result  in  an  increase  in  their  recall  performance. 

Summary  of  the  Findings 

Table  28  serves  to  summarize  the  findings  of  this  research  effort  and  is  useful  in 
constructing  a  comprehensive  interpretation  of  the  results. 

Table  28. 

Findings  summary. 


Research 

Question/Answer 

Tests  Used 

Results 

(N.S.  =  not  significant) 

1 .  Did  IPS  improve 
retentionyrecall  over 

Between  Methods: 

EP  Knowledge  D 1 

Significant  difference:  Traditional 

traditional  method  in  the 

Pre  to  D 1  Post 

method  superior. 

overall  population? 

No,  but  no  different  from 

EP  Knowledge  D 1 

N.S. 

traditional  from  Day  1 
Posttest  to  Day  7  Posttest. 

Post  to  D7  Post 

Within  Methods: 

EP  Knowledge  D 1 
Post  to  D7  Post 

Both  methods  showed  significant 
performance  gains. 

Note:  D  =  Day;  EP  =  emergency  procedures;  IV 

G  =  membership  group. 
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Table  28  (continued). 
Findings  summary. 


Research 

Questions/Answers 

Tests  Used 

Results 

(N.S.  =  not  significant) 

2.  Is  there  a  difference 

EP  Knowledge  D 1 

MG  3  treatment  and  control  groups’ 

in  performance  between 

Pre  to  D 1  Post 

performance  gains  sig.  diff.  than  other 

the  traditional  and 
using  the  IPS  methods 

groups,  but  no  method  superior. 

with  respect  to 
experience  levels? 

Yes. 

EP  Knowledge  D 1 

MG  2  control  group  and  MG  3 

Post  to  D7  Post 

treatment/control  groups’ 
performance  gains  sig.  diff.  than  other 

groups,  but  IPS  not  clearly  superior. 

3.  Did  learning  style 

EP  Knowledge  D 1 

N.S. 

preference  (ESP)  affect 

Pre  to  D 1  Post  / 

performance? 

ESP  Questionnaire 

No. 

EP  Knowledge  D 1 
Post  to  D7  Post  / 

ESP  Questionnaire 

N.S. 

4.  Did  performance 

EP  Knowledge  D 1 

N.S. 

affect  subjective 
assessment  (SA)  of  IPS? 

Post  /  SA  Survey 

EP  Knowledge  D7 

N.S. 

No  correlation  found. 

Post  /  SA  Survey 

EP  Knowledge  D 1 
to  D7  Post  /  SA 
Survey 

N.S. 

5.  Did  learning  style 
preference  affect 
subjective  assessment  of 
IPS? 

ESP  /  SA  Survey 

N.S. 

No  correlation  found. 

6.  Does  experience  affect 

Membership  Group 

Significant  differences  in  assessments 

subjective  assessment? 

Yes.  Degree  of  positive 
assessment  is  inversely 
related  to  amount  of 
experience. 

/  SA  Survey 

of  IPS  with  regard  to  experience 
levels. 

Note:  D  =  Day;  EP  =  emergency  procedures;  MG  =  membership  group. 
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Table  28  (continued). 
Findings  summary. 


Research 

Question/Answer 

Tests  Used 

Results 

(N.S.  =  not  significant) 

7.  Is  IPS  positively 
assessed  by  the  majority 
of  participants? 

SA  Survey 

96%  rate  IPS  in  a  positively-inclined 
manner. 

Yes.  Overall  positive 
assessment  by  majority. 

8.  Are  the  IPS  symbols 
intuitive  and  memorable? 

Yes. 

Symbol 

Recognition  D 1  to 

D7  Posttests 

Average  of  the  least-experienced 
participants  (MG  2  and  MG  3)  scored 
nearly  the  same  or  better  than  the 
highly-experienced  group  (MG  1). 

9.  Is  there  an  association 
between  study  time  and 
performance? 

Y es,  but  for  MG  1  only. 

Study  Experience 
Report  /  EP 
Knowledge  & 
Symbol 

Recognition  D7 
Posttests 

Significant  difference:  MG  1 
treatment  and  control  groups’  study 
time  has  significant  correlation  with 
performance  gains.  Note:  MG  2 
control  group  studies  significantly 
more  (3x)  than  all  other  groups. 

Note:  D  =  Day;  EP  =  emergency  procedures;  IV 

G  =  membership  group. 

Limitations 


Study  population 

The  following  discussion  is  based  on  the  observations  of  the  researcher  and  may  provide 
useful  insights  into  the  limitations  of  using  this  study  population  to  assess  training  methods. 
Aviators  are  professionals  who  are  proud,  egotistical,  and  defensive  of  their  knowledge  and 
skills.  Hence,  it  was  apparent  during  the  recruitment  of  participants  that  there  was  a  general 
aversion  to  tests  that  might  challenge  or  question  their  knowledge.  This  was  especially  true  of 
the  highly-experienced  pilots  who  were  particularly  reluctant  to  participate  in  the  study  when 
they  learned  of  the  testing  details.  The  minimally-experienced  pilots  were  similarly  reluctant, 
but  not  to  the  degree  of  the  more  experienced  pilots.  Those  in  the  minimally-experienced  group 
seemed  to  have  hopes  of  some  potential  benefit  from  the  experience,  whereas,  the  highly- 
experienced  saw  little  or  no  benefit  from  participation.  On  the  other  hand,  the  naive  population 
awaiting  the  start  of  flight  school  was  eager  to  take  part  in  the  study.  It  was  clear  that  they 
perceived  no  threat  to  their  aviator  status  (since  they  did  not  have  one)  and  that  there  was  great 
potential  for  personal  benefit  from  their  experience.  It  may  be  because  of  this  seeming  threat  to 
status  and  the  perceived  potential  for  individual  benefit  that  there  were  palpable  differences 
between  experience  groups  and  individuals  in  their  motivation  to  support  the  research  effort. 
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In  addition,  flying  helicopters,  whether  teaching  or  learning,  is  a  very  demanding 
enterprise.  Obviously,  anything  relating  to  the  successful  accomplishment  of  flight  school- 
related  tasks  or  duties  took  priority  over  the  voluntary  participation  in  this  research  project. 

Thus,  varying  degrees  of  commitment  to  the  research  project  and  to  learning  and  using  the  IPS 
symbols  were  observed.  This  variation  in  commitment  was  corroborated  by  the  occasional 
difficulty  in  scheduling  the  follow-on  (Day  7)  testing. 

The  highly-experienced  group  was  witnessed  (through  verbal  comments  to  the  researcher 
and  to  each  other)  to  be  especially  partial  to  their  own  memorization  techniques.  This  may  have 
negatively  influenced  their  commitment  to  adding  the  IPS  as  a  mnemonic  strategy  fearing 
interference  with  their  own  systems.  This  is  quite  understandable  since  members  have  years  of 
successful  experience  using  their  own  strategies. 

Length  of  study  period 

Due  to  limited  resources  in  time  and  funding,  the  study  was  designed  to  measure  the 
performance  gains  of  at  least  ninety  participants  over  a  one  week  period.  Since  memorization 
strategies  such  as  the  IPS  are  used  to  enhance  long  term  memory,  longitudinal  studies  over  much 
longer  periods  may  provide  a  much  clearer,  substantive  depiction  of  the  effectiveness  of  the  IPS 
compared  to  that  of  the  tradition  method. 

Control  of  participant  study  time 

As  mentioned  several  times  previously,  due  to  participant  circumstances  (involvement  in 
Army  flight  school),  the  amount  of  participant  study  time  could  not  be  manipulated  (mandated, 
limited,  or  controlled)  in  any  way  by  the  researcher.  Each  group  member  was  encouraged  to 
study  their  emergency  procedures  using  their  assigned  method,  and  study  time  data  was  collected 
which  showed  that  study  time  sometimes  varied  by  group  assignment.  As  in  any  academic 
environment,  time  committed  to  study  will  vary  with  each  individual  as  it  did  in  this  research 
effort.  Although  unfavorable  to  the  experimental  research  effort,  the  variability  of  study  time 
may  have  produced  findings  that  are  more  representative  of  what  might  be  experienced  in  the 
real  world  applications  of  these  methods.  That  is,  average  study  time  varies  from  one  group  to 
the  next. 

Ceiling  effect 

Regarding  the  performance  of  the  highly-experienced  membership  group,  it  is  possible 
that  treatment  effects  on  knowledge  performance  were  obscured  by  a  ceiling  effect  caused  by  the 
level  of  preexisting  knowledge  (the  dependent  variable)  of  this  group. 

Study  implications 

This  research  examined  the  utility  and  merit  of  a  novel  system  of  intuitive  pictures  (the 
Intuitive  Pictorial  System  or  IPS)  taught  as  a  mnemonic  strategy  to  facilitate  learning  and 
accurate  recall  of  complex  emergency  procedures.  Much  research  has  been  conducted  (Chapter 
3)  on  visual  memory  and  memorization  strategies;  however,  a  review  of  the  literature  found  no 
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evidence  of  any  attempts  to  introduce  any  structured,  formal  mnemonic  strategy  system  to  the 
learning  of  Army  aviation  emergency  procedures.  It  is  hoped  that  these  findings  will  contribute 
to  the  body  of  knowledge  and  provide  important  information  about  the  practical  application  of  a 
pictorial  memorization  strategy  from  which  informed  resource  allocation  decisions  can  be  made. 

The  positive  results  of  the  Subjective  Assessment  surveys  and  Symbol  Recognition  tests 
indicated  the  popularity,  intuitiveness,  and  memorability  of  the  prototype  symbols.  Information 
from  this  research  can  provide  guidance  on  the  refinement  and  integration  of  a  pictorial  system 
or  systems  which  could  augment  textual  emergency  procedures  and  abbreviate  checklists  and 
aviation  cockpit  instrumentation.  This  information  could  lead  to  important  innovations  to 
current  U.S.  Army  teaching  techniques  and  minimize  the  use  of  random  memorization 
techniques  by  students.  Applications  of  a  pictorial  system  representing  common  aviation 
procedures  could  be  of  benefit  to  the  other  military  services  and  the  national  and  international 
civil  aviation  communities  in  much  the  same  way  as  international  highway  signs  have  become. 
A  standardized  pictorial  system  of  aviation  procedures  may  not  just  benefit  individual  graduate 
and  student  pilots,  but  could  be  a  valuable  contribution  to  aviation  safety. 

Conclusions 


The  goal  of  this  research  project  was  to  determine  the  utility  and  merit  of  a  novel  system 
of  intuitive  pictures  (the  IPS)  taught  as  a  mnemonic  strategy  to  facilitate  the  learning  and 
accurate  recall  of  complex  aviation  emergency  procedures.  Although  the  study’s  findings  did 
not  show  that  the  IPS  produced  performance  gains  superior  to  that  of  the  traditional  method,  the 
findings  did  demonstrate  the  utility  and  merit  of  the  system  as  an  augmentation  to  traditional 
textual  procedures.  This  was  evidenced  by  user  assessments,  user  comments,  and  symbol 
recognition  test  performance.  The  manner  in  which  the  symbols  were  able  to  facilitate  the  recall 
of  uncommon,  unfamiliar  terms  and  phrases  in  a  naive  population  to  a  level  comparable  to  that 
of  highly-experienced  pilots  in  just  one  week,  highlights  the  potential  for  such  a  mnemonic 
strategy  to  aid  in  the  encoding  of  information  into  long-term  memory.  The  promise  in 
incorporating  a  refined  IPS  into  the  U.S.  Army  flight  educational  program  would  be  in  reducing 
the  expense  and  time  it  takes  to  teach,  learn,  and  maintain  complex  aviation  emergency 
procedures,  thus,  enhancing  aviation  safety  and  preserving  vital  resources. 

Suggestions  for  future  research 

The  objective  findings  of  this  study  did  not  achieve  the  clear  results  of  previous  research 
which  reported  that  pictorial  mnemonics  lead  to  benefits  beyond  simple  rote  memory  (Carney 
and  Levin,  2003;  Cherry,  Dokey,  Reese,  and  Brigman,  2003).  The  limitations  outlined  above 
may  have  unintentionally  masked  or  suppressed  the  objective  measures  when  comparing  the  IPS 
to  the  traditional  training  method.  Research  is  needed  which  provides  for  a  more  definitive 
assessment  of  the  IPS’s  capacity  to  improve  the  long-term  memory  of  aviators  for  emergency 
procedures.  Future  research  should  employ  designs  which  contend  with  the  cited  limitations  and 
should  attempt  to  validate  the  findings  of  this  research  effort  using  other  designs  capable  of 
strengthening  construct  validity. 
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A  course  for  improving  the  strength  and  eertitude  of  this  study’s  eonelusions  regarding 
the  IPS’s  effeetiveness  would  be  to  foeus  the  researeh  on  a  naive  group  over  a  mueh  longer 
period  of  time.  In  all  likelihood,  a  naive  population  would  provide  the  best  indieators  of  training 
method  effectiveness  beeause,  as  observations  during  this  study  revealed,  populations  with 
previous  experienee  also  possess  biases  and  motivational  issues  that  are  difficult  to  control, 
influence,  and/or  overeome.  A  naive  population  seems  to  possess  an  initial  willingness  to  learn 
in  the  manner  in  whieh  the  material  is  presented  and  would  therefore,  provide  a  elearer,  more 
objective  indieation  of  method  effeetiveness.  By  extending  the  study  period  over  a  longer  period 
of  time  (six  months  to  several  years),  the  effectiveness  of  each  method  for  maintaining  or 
improving  aviators’  long-term  memory  for  emergeney  proeedures  eould  be  deeisively 
determined. 

That  said,  it  is  suggested  that  a  future  study  be  eonducted  which  randomly  assigns  at  least 
six  elasses  of  20  new  flight  students  to  reeeive  either  the  IPS  or  traditional  EP  training  method. 
As  the  students  progress  through  the  required  EP  training,  their  retention  of  the  EPs  would  be 
tested  at  three-week  intervals  until  the  eompletion  of  flight  school  36-weeks  later,  thus, 
providing  12  observations.  Comparisons  of  IPS  and  traditionally-trained  performances  over  this 
extended  period  would  provide  a  better  indieator  of  differenees  in  training  method  effeetiveness, 
thus  strengthening  internal  and  external  validity.  The  methodology  outlined  above  would 
provide  the  foundation  for  a  longitudinal  study  whieh  eould  remain  ongoing  for  a  number  of 
years.  Upon  graduation,  aviators  are  easily  traeked  from  posting  to  posting.  Eollow-on  data 
eolleetion  (memory  tests  and  surveys  via  personal  or  internet  interview)  would  provide  valuable 
information  as  to  the  eapaeity  of  eaeh  training  method  to  enhanee  long-term  memory  and  reeall 
aecuracy. 

It  is  unreasonable  to  expeet  the  administrators  of  the  Army’s  flight  training  program  to 
allow  the  eontrol  (by  mandate  or  restriction)  of  the  time  flight  students  spend  studying  EPs. 
Therefore,  in  order  to  provide  analysis  of  study  time  effeets,  it  is  reeommended  that  study  time 
be  reeorded  as  reported  by  eaeh  partieipant.  At  the  eompletion  of  the  data  eolleetion, 
partieipants  of  the  IPS  and  traditional  groups  would  be  matehed  or  paired  by  speeifie  study  time 
or  study  time  periods  (as  in,  0-1  hour  per  week,  2-4  hours  per  week,  ete.)  and  eomparisons  made 
with  study  time  as  the  factor.  Conceivably,  this  approaeh  will  yield  the  effeets,  differences, 
and/or  assoeiations  of  study  time,  training  method,  and  reeall  ability  over  the  mueh  longer  data 
eolleetion  period.  Einally,  should  future  researeh  involve  partieipants  of  different  experienee 
levels,  the  design  should  remove  the  potential  for  eeiling  effeets  by  making  the  experimental 
tasks  mueh  more  difficult  than  posed  in  this  study,  espeeially  for  the  highly-experieneed  group. 

Eurther  researeh  into  mnemonic  strategies  in  the  aviation  eontext  should  not  be  limited  to  that  of 
the  assessment  of  the  IPS.  Advanees  in  teehnology,  coekpit  displays,  and  eomputer-generated 
human  interfaees  are  ehanging  the  manner  in  whieh  the  military  pilot  interaets  with  the  aireraft. 
These  advanees  often  ereate  new  operational  requirements  whieh  inerease  the  pilot’s  eognitive 
workload.  Any  mental  strategy  whieh  ean  reduee  or  faeilitate  the  pilot’s  inereasing  eognitive 
workload  should  be  investigated.  As  May  and  Kahnweiler  (2000)  write,  “The  solution  may  lie 
in. .  .finding  creative  ways  to  devote  more  partieipant  training  time  to  those  skill  sets  that  really 
matter  to  the  strategie  direetion  of  the  organization.” 
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Appendix  A. 


Intuitive  Pietorial  System  symbols  and  rules. 


Basie  Aireraft  Systems  Symbols 


Engine  Systems 


Power  Control  Lever  Positions 


E 


Single  Engine 


Dual  Engine 


V 


ECU  Loekout 


APU  (Auxiliary 
Power  Unit) 


Plight  Controls 


Cyelie 


=  Colleetive 


Puel  Seleetor  Positions 


Crossfeed 


=  Direet 


9 


=  Pedals 
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bu  K 


Main  Rotor 


External  Cargo/Stores 


Tail  Rotor 


Transmission  Modules: 


R 

1  1  |m  1 

LH  Aeeessory  LH  Input  Main  RH  Input  RH  Aeeessory 


Fuel  Boost  Pump 


Fuel  Pump  -  APU  Boost 


Basic  Action  Symbols 


System  Shutdown: 


=  Intentional  System  Shutdown  (plaeed  over  the 
appropriate  symbol) 


Henee: 


=  Emergeney  (Single)  Engine  Shutdown 


=  Emergeney  Dual  Engine  Shutdown 
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I  I  I  I  I  I  ■Mill  ■!  I  I  I  L 


=  APU  Shutdown 


A 


“Forward”  or  “Increasing” 

“Retard”  or  “Aft”  or  “Deereasing” 

“Adjust” 

“Up”  or  “Inerease” 

“Down”  or  “Reduce”  or  “Deerease”  or  “Jettison” 

“Adjust” 

Henee: 

=  Forward  Cyclic 

=  Aft  Cyclic 

=  Cyclic  Adjust 
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=  Decreasing 
Rotor 


=  Increasing 
Rotor 


=  Apply  Pedal 
Force 


=  External  Cargo/ 
Stores  -  Jettison 


Landings: 


$ 


=  Autorotate 


=  Land  as  Soon  as  Possible 


=  Land  as  Soon  as  Practicable 


Engine  Power 
Control  Lever  - 
Retard 


Engine  Power 
Control  Lever  - 
Adjust 


Establish  Single 
Engine  Airspeed 


Refer  To  Single  Engine 
Eailure  Emergency 
Procedures 


69 


Situational  Symbols 


Failures: 


=  System  Failure  when  plaeed  over  the  appropriate 
symbol. 

=  Chip  when  plaeed  within  the  symbol. 

Henee: 


=  Single  Engine  Failure 


=  Dual  Engine  Failure 


=  APU  Failure 


Tail  Rotor  Failure  or  Foss  of  Tail  Rotor  Thrust 


Chips: 


Chip  in  the  Main  Transmission 


R  I  X  I  H 


Engine  Chip 


Chips  in  the  FH  and 
RH  Input  Modules 


Chip  in  the  FH 
Input  Module 


Chip  in  the  RH 
Input  Module 


R|Xlfn|XlR 

R  IXlm  1  1  |r 

R  i|m|XlR 
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Chips  in  the  LH  and 
RH  Accessory  Modules 


Chip  in  the  LH 
Accessory  Module 


Chip  in  the  RH 
Accessory  Module 


Other  Situational  Symbols: 


=  Engine  Compressor  Stall  (engine  symbol  on  its  side) 


=  Engine  High  Speed  Shaft  Eailure 


Tail  Rotor  Quadrant  Eailure  (loss  of  control) 


=  Eightning  Strike 


General  Rules 


1 .  Always  use  right  seat  perspeetiye. 

2.  An  arrow  pointing  to  the  right  depicts  “increase,”  an  arrow  pointing  to  the  left  depicts 
“decrease,”  and  an  arrow  pointing  left  AND  right  indicates  “adjust”  (either  increase  or 
decrease,  or  both). 

3.  The  large  symbol  is  the  situation,  the  smaller  symbols  are  the  proeedural  steps. 

4.  “E”  refers  to  engine,  “T”  refers  to  main  rotor,  and  “/”  refers  to  shutdown. 

5.  “X”  oyer  the  symbol  depicts  failure,  while  “x”  within  the  symbol  depicts  a  chip. 

6.  When  using  the  IPS  to  remember  emergency  procedures,  try  to  imprint  the  whole  image 
of  a  proeedure  (the  situation  symbol  and  step  symbols  together)  in  your  memory  as  one 
pieture. 
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Appendix  B. 

Emergency  Procedures  Knowledge  Test. 


Participant  # _ . 

1 .  What  is  the  first  step  in  the  event  of  an  engine  compressor  stall? 


2.  What  is  the  last  step  in  the  event  of  a  loss  of  tail  rotor  thrust  when  not  operating  at  low 
airspeed  or  at  a  hover? 


3.  What  is  the  emergency  procedure  for  a  tail  rotor  quadrant  caution  light  with  an  accompanying 
loss  of  tail  rotor  control? 


4.  What  are  the  steps  of  an  emergency  engine  shutdown? 


5.  During  a  single-engine  failure,  if  continued  flight  is  possible,  what  two  steps  should  be 
performed  after  adjusting  the  collective  and  jettisoning  the  external  stores? 


6.  If  %  RPM  R  is  increasing  and  the  engine  does  not  respond  to  engine  power  control  lever 
movement  between  Fly  and  Idle,  what  steps  should  be  taken  after  establishing  single  engine 
airspeed? 


7.  What  should  be  done  in  the  event  of  a  dual-engine  failure? 


8.  In  the  event  of  a  high-speed  shaft  failure,  what  is  Step  2  and  what  is  Step  4? 
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9.  What  is  the  first  thing  to  do  when  tail  rotor  thrust  is  lost  while  at  a  hover? 


10.  When  should  you  land  when  %  RPM  R  is  deereasing? 


11.  In  the  event  of  a  ehip  in  the  right  hand  (RH)  input  module,  what  step(s)  must  be  performed? 


12.  An  engine  ehip  caution  light  appears  for  the  #1  engine.  What  are  the  three  numbered  steps 
in  the  procedure? 


13.  You  must  be  prepared  to  perform  what  steps  following  a  lightning  strike? 


14.  A  chip  light  appears  indicating  a  chip  in  the  main  transmission.  What  must  you  do? 


15.  When  should  the  engine  power  control  levers  be  moved  to  the  OFF  position  during  a  loss  of 
tail  rotor  thrust  while  at  a  hover? 
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Appendix  C. 

Emergency  Procedures  Knowledge  Test  answer  sheet. 


Participant  # _ . 

1 .  What  is  the  first  step  in  the  event  of  an  engine  compressor  stall? 

Collective  -  Reduce 

2.  What  is  the  last  step  in  the  event  of  a  loss  of  tail  rotor  thrust  when  not  operating  at  low 
airspeed  or  at  a  hover? 


ENG  POWER  CONE  levers  -  OEE  (when  intended  point  of  landing  is  assured). 

3.  What  is  the  emergency  procedure  for  a  tail  rotor  quadrant  caution  light  with  an  accompanying 
loss  of  tail  rotor  control? 

1 .  Collective  -  Adjust. 

2.  EAND  AS  SOON  AS  PRACTICABEE. 

4.  What  are  the  steps  of  an  emergency  engine  shutdown? 

1.  ENG  POWER  CONT  lever(s)  -  OEE. 

2.  ENG  EUEE  SYS  selector(s)  -  OEE. 

3.  EUEE  BOOST  PUMP  CONTROE  switch(es)  -  OEE. 

5.  During  a  single-engine  failure,  if  continued  flight  is  possible,  what  two  steps  should  be 
performed  after  adjusting  the  collective  and  jettisoning  the  external  stores? 

4.  Establish  single-engine  airspeed. 

5.  EAND  AS  SOON  AS  PRACTICABEE. 

6.  If  %  RPM  R  is  increasing  and  the  engine  does  not  respond  to  engine  power  control  lever 
movement  between  Ply  and  Idle,  what  steps  should  be  taken  after  establishing  single  engine 
airspeed? 

4.  Perform  EMERG  ENG  SHUTDOWN  (affected  engine). 

5.  Refer  to  single  engine  failure  emergency  procedures. 

7.  What  should  be  done  in  the  event  of  a  dual-engine  failure? 

AUTOROTATE 
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8.  In  the  event  of  a  high-speed  shaft  failure,  what  is  Step  2  and  what  is  Step  4? 

2.  Establish  single  engine  airspeed. 

4.  Refer  to  single  engine  failure  emergeney  proeedures. 

9.  What  is  the  first  thing  to  do  when  tail  rotor  thrust  is  lost  while  at  a  hover? 

Colleetive  -  Reduee 

10.  When  should  you  land  when  %  RPM  R  is  deereasing? 

Land  as  soon  as  praetieable. 

11.  In  the  event  of  a  ehip  in  the  right  hand  (RH)  input  module,  what  step(s)  must  be  performed? 

1 .  ENG  POWER  CONE  lever  on  affeeted  engine  -  IDLE. 

2.  LAND  AS  SOON  AS  POSSIBLE. 

12.  An  engine  ehip  eaution  light  appears  for  the  #1  engine.  What  are  the  three  numbered  steps 
in  the  proeedure? 

1 .  ENG  POWER  CONTROL  lever  -  Retard  to  reduee  torque  on  affeeted  engine. 

2.  EMER  ENG  SHUTDOWN  (affeeted  engine). 

3.  Refer  to  single  engine  failure  emergeney  proeedures. 

13.  You  must  be  prepared  to  perform  what  steps  following  a  lightning  strike? 

1 .  ENG  POWER  CONT  levers  -  Adjust  as  required  to  eontrol  %  RPM. 

2.  LAND  AS  SOON  AS  POSSIBLE. 

14.  A  ehip  light  appears  indieating  a  ehip  in  the  main  transmission.  What  must  you  do? 

LAND  AS  SOON  AS  POSSIBLE 

15.  When  should  the  engine  power  eontrol  levers  be  moved  to  the  OLE  position  during  a  loss  of 
tail  rotor  thrust  while  at  a  hover? 

(5  to  10  feet  above  touehdown) 
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Appendix  D. 

Subjective  Assessment  Survey. 

Participant  # _ 

Please  circle  the  responses  that  most  accurately  answer  the  following  questions. 

CIRCLE  ONLY  ONE  ANSWER  PER  QUESTION. 

1 .  I  found  that  the  Intuitive  Pictorial  System  (IPS)  was  not  a  useful  tool  (strategy)  for 
memorizing  the  aircraft’s  emergency  procedures. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

2.  I  prefer  my  own  system  of  memorization  over  that  of  the  IPS. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

3.  If  given  the  choice,  I  would  not  use  the  IPS  in  the  future. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

4.  I  believe  the  IPS  just  adds  to  the  amount  of  material  I  already  have  to  learn  and  is,  thus,  an 
additional  burden. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

5.  Generally  speaking,  the  symbols  used  in  the  IPS  were  not  intuitive  to  me  (easily  recognized 
without  evident  thinking). 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

6.  I  believe  the  IPS  is  too  complex  a  system  to  benefit  the  average  new  aviation  student. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

7.  I  believe  the  IPS  has  not  improved  my  ability  to  recall  emergency  procedures  when  needed. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 
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8.  I  don’t  believe  that  the  IPS  should  be  taught  to  future  aviation  students. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

9.  The  IPS  did  not  assist  me  in  memorizing  the  required  emergeney  proeedures. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

10. 1  eonsider  learning  the  IPS  to  be  a  waste  of  valuable  time  that  I  eould  have  used  memorizing 
the  emergeney  proeedures  my  own  way. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

1 1 . 1  would  not  reeommend  integrating  and  employing  the  IPS  as  part  of  the  U.  S.  Army’s 
offieial  flight  training  syllabus. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

12. 1  believe  the  IPS  may  eause  eonfusion  to  the  majority  of  pilots. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

13.  The  IPS  symbols  should  not  be  used  in  an  aireraft  emergeney  eheeklist  for  use  in  the  eoekpit 
instead  of  written  text. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

14. 1  believe  the  IPS  eould  aetually  eause  or  eontribute  to  eonfusion  during  an  aetual  aireraft 
emergeney  situation. 

Strongly  agree  Agree  Neutral  Disagree  Strongly  disagree 

15.  Below  please  provide  any  additional  eomments  regarding  the  IPS  and  its  symbols; 
i.e.,  speeifie  symbols  that  were  or  were  not  partieularly  intuitive,  speeifie  symbols 
whieh  need  refinement  or  ehange,  reeommendations  for  future  applieations. 
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Appendix  E. 

Subjective  Assessment  Survey  scoring  procedures. 


•  Answers  to 
Strongly  agree 
Agree 
Neutral 
Disagree 
Strongly  disagree 


the  Subjective  Assessment  Surveys  will  be  scored  as  such; 
=  5  points 
=  4  points 
=  3  points 
=  2  points 
=  1  point 


•  Total  scores  will  be  converted  to  the  following  classifications  (groups): 
14-28  =  positive  assessment 
29-41  =  positive  to  neutral  assessment 
42  =  neutral  assessment 
43  -  55  =  neutral  to  negative  assessment 
56  -  70  =  negative  assessment 
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11. 


13. 


R  [X]ni  I  I  |n 


14. 


1 1 1 1 1 1 


I 


15. 


18. 


19. 


20. 


Appendix  G. 

Symbol  Recognition  Test  answer  sheet. 


Participant  # _ . 

Identify  each  of  the  following  symbols  by  writing  in  the  aircraft  system,  situation,  or  action 
symbolized  by  it. 


1. 


=  Collective  Reduce 


2. 


=  Establish  Single  Engine  Airspeed 


3.  =  Dual  Engine  Eailure 


4.  =  Decreasing  Rotor 


5.  =  Engine  Compressor  Stall 

6.  =  Crossfeed 


7^" 

7.  =  External  Cargo/Stores 
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8. 


=  Fuel  Boost  Pump 


9.  =  Forward  Cyclic 

t 

10.  =  Engine  Power  Control  Lever  -  Adjust 


11. 


=  Land  as  Soon  as  Possible 


12. 


=  Refer  To  Single  Engine  Lailure  Emergency  Procedures 


13. 


R  [X]ni  I  I  |r 


=  Chip  in  the  EH  Input  Module 


14.  =  Tail  Rotor  Eailure  or  Loss  of  Tail  Rotor  Thrust 

15.  =  Collective  Adjust 


83 


16. 


=  Emergency  (Single)  Engine  Shutdown 


E 

'V 

17.  =  Engine  Power  Control  Eever  EE Y 

18.  =  Engine  Chip 


19.  =  Intentional  System  Shutdown  (placed  over  the  appropriate 
symbol) 

20.  =  Engine  Power  Control  Eever  -  Retard 
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Appendix  H. 


The  VARK  Questionnaire. 

Partieipant  # _ .  Revised  version  -  January  2005. 

This  questionnaire  aims  to  find  out  something  about  your  preferenees  for  the  way  you  work  with 
information.  You  will  have  a  preferred  learning  style  and  one  part  of  that  learning  style  is  your 
preferenee  for  the  intake  and  output  of  ideas  and  information. 

Choose  the  answer  whieh  best  explains  your  preferenee  and  eliek  on  the  box  next  to  the  letter. 
Please  seleet  more  than  one  response  if  a  single  answer  does  not  mateh  your  pereeption. 

Leave  blank  any  question  whieh  does  not  apply. 

1 .  You  are  about  to  give  direetions  to  a  person  who  is  standing  with  you.  She  is  staying 
in  a  hotel  in  town  and  wants  to  visit  your  house  later.  She  has  a  rental  ear.  I  would: 

a.  draw  or  provide  a  map  on  paper. 

b.  tell  her  the  direetions. 

e.  write  down  the  direetions  (without  a  map). 

d.  eolleet  her  from  the  hotel  in  a  ear. 

2.  You  are  not  sure  whether  a  word  should  be  spelled  'dependent'  or  'dependant'.  I 
would: 

a.  look  it  up  in  the  dietionary. 

b.  see  the  word  in  my  mind  and  ehoose  by  the  way  it  looks. 

e.  sound  it  out  in  my  mind. 

d.  write  both  versions  down  on  paper  and  ehoose  one. 

3.  You  have  just  reeeived  a  eopy  of  your  itinerary  for  a  world  trip.  This  is  of  interest  to 
some  friends.  I  would: 

a.  phone,  text  or  email  them  and  tell  them  about  it. 

b.  send  them  a  eopy  of  the  printed  itinerary. 

e.  show  them  on  a  map  of  the  world. 

d.  deseribe  what  I  plan  to  do  at  eaeh  plaee  on  the  itinerary. 

4.  You  are  going  to  eook  something  as  a  speeial  treat  for  your  family.  I  would: 

a.  eook  something  familiar  without  the  need  for  instruetions. 

b.  thumb  through  the  eookbook  looking  for  ideas  from  the  pietures. 

e.  refer  to  a  speeific  eookbook  where  there  is  a  good  reeipe. 
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5.  A  group  of  tourists  has  been  assigned  to  you  to  find  out  about  wildlife  reserves  or 
parks.  I  would; 

a.  drive  them  to  a  wildlife  reserve  or  park. 

b.  show  them  slides  and  photographs. 

c.  give  them  pamphlets  or  a  book  on  wildlife  reserves  or  parks. 

d.  give  them  a  talk  on  wildlife  reserves  or  parks. 

6.  You  are  about  to  purehase  a  new  CD  player.  Other  than  priee,  what  would  most 
influenee  your  deeision? 

a.  The  salesperson  telling  you  what  you  want  to  know. 

b.  Reading  the  details  about  it. 

e.  Playing  with  the  eontrols  and  listening  to  it. 
d.  It  looks  really  smart  and  fashionable. 

7.  Reeall  a  time  in  your  life  when  you  learned  how  to  do  something  like  playing  a  new 
board  game.  Try  to  avoid  choosing  a  very  physical  skill,  e.g.  riding  a  bike.  I  learned 
best  by: 

a.  visual  clues  —  pictures,  diagrams  and  charts. 

b.  written  instructions. 

c.  listening  to  somebody  explaining  it. 

d.  doing  it  or  trying  it. 

8.  You  have  a  knee  problem.  I  would  prefer  that  the  doctor: 

a.  told  me  what  was  wrong. 

b.  showed  me  a  diagram  of  what  was  wrong. 

c.  used  a  model  of  a  knee  to  show  me  what  was  wrong. 

9.  You  are  about  to  learn  to  use  a  new  program  on  a  computer.  I  would: 

a.  sit  down  at  the  keyboard  and  experiment  with  the  program. 

b.  read  the  manual  that  came  with  the  program. 

c.  telephone  or  text  a  friend  and  ask  questions  about  the  program. 

10.  You  are  staying  in  a  hotel  and  have  a  rental  car.  You  would  like  to  visit  friends  whose 
address/location  you  do  not  know.  I  would  like  them  to: 

a.  draw  me  a  map  on  paper  or  provide  a  map  from  the  internet. 

b.  tell  me  the  directions. 

c.  write  down  the  directions  (without  a  map). 

d.  collect  me  from  the  hotel  in  a  car. 
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1 1 .  Apart  from  the  price,  what  would  most  influence  your  decision  to  buy  a  particular 
textbook? 

a.  I  have  used  a  copy  before. 

b.  A  friend  talking  about  it. 

c.  Quickly  reading  parts  of  it. 

d.  The  way  it  looks  is  appealing. 

12.  A  new  movie  has  arrived  in  town.  What  would  most  influence  your  decision  to  go 
(or  not  go)? 

a.  I  heard  a  review  about  it  on  the  radio. 

b.  I  read  a  review  about  it. 

c.  I  saw  a  preview  of  it. 

13.  Do  you  prefer  a  teacher  who  likes  to  use: 

a.  a  textbook,  handouts  and  readings. 

b.  flow  diagrams,  charts  and  graphs. 

c.  field  trips,  models,  laboratories  and  practical  sessions. 

d.  class  or  email  discussion,  online  chat  groups  and  guest  speakers. 

Copyright  Version  7.0  (2006)  held  by  Neil  D.  Fleming,  Christchurch,  New  Zealand  and  Charles 
C.  Bonwell,  Green  Mountain  Falls,  Colorado  80819  U.S.A. 
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Appendix  L 
VARK  scoring  chart. 

Use  the  following  scoring  chart  to  find  the  VARK  category  that  each  of  the  answers 
corresponds  to.  Circle  the  letters  that  correspond  to  the  answers,  e.g.,  if  question  #3  was 
answered  b  and  c,  circle  R  and  V  in  the  question  3  row. 


Question 

a  category 

b  category 

c  category 

d  category 

3 

A 

© 

0 

K 

Scoring  chart: 


Question 

a  category 

b  category 

c  category 

d  category 

1 

V 

A 

R 

K 

2 

R 

V 

A 

K 

3 

A 

R 

V 

K 

4 

K 

V 

R 

5 

K 

V 

R 

A 

6 

A 

R 

K 

V 

7 

V 

R 

A 

K 

8 

A 

V 

K 

9 

K 

R 

A 

10 

V 

A 

R 

K 

11 

K 

A 

R 

V 

12 

A 

R 

V 

13 

R 

V 

K 

A 

88 


Calculating  the  scores: 


Count  the  number  of  eaeh  of  the  VARK  letters  that  have  been  eireled  to  get  the  seore  for  eaeh 
VARK  eategory. 


Total  number  of  Vs  eireled  = 


Total  number  of  As  eireled  = 


Total  number  of  Rs  eireled  = 


Total  number  of  Ks  eireled  = 


Calculating  the  preferences  /  Scoring  instructions: 

Because  respondents  can  choose  more  than  one  answer  for  each  question  the  scoring  is  complex. 
It  can  be  likened  to  a  set  of  four  stepping-stones  across  water. 

1 .  Add  up  the  scores,  V  +  A  +  R  +  K  = _ . 

2.  Enter  the  scores  from  highest  to  lowest  on  the  stones  below,  with  their  V,  A,  R,  and  K 
labels. 

3.  The  stepping  distance  comes  from  the  following  table: 


The  total  of  the  four  VARK  scores  is 

The  stepping  distance  is 

10-16 

1 

17-22 

2 

23-26 

3 

More  than  26 

4 

4.  The  first  preference  is  the  highest  score,  so  check  the  first  stone  as  one  of  the 
preferences  and  enter  its  label  on  the  stone. 

5.  If  the  next  stone  can  be  reached  with  a  step  equal  to  or  less  than  the  stepping  distance, 
then  check  the  next  one  also. 


Scoring  VARK  Stepping-Stones: 


Once  the  next  stone  eannot  be  reaehed,  the  respondent’s  set  of  preferenees  have  been  defined.  If 
there  is  more  than  one  preferenee  ehecked,  the  respondent  has  multimodal  preferenees.  If  only 
the  first  stone  is  checked,  then  the  respondent  has  a  single  preference. 

For  those  with  a  Single  Preference: 

If  a  respondent  has  a  single  preferenee  (V,  A,  R,  or  K),  the  preferenee  is  mild,  strong,  or  very 
strong.  Use  the  table  below  (Result  Table)  to  find  this  out. 

1.  Use  the  total  number  of  responses  (from  step  1  above).  This  is  used  for  finding  the  plaee 
in  Column  One.  Read  across  from  the  appropriate  line  in  Column  One. 

2.  Now  take  the  difference  between  the  highest  score  and  the  next  highest  seore.  Read 
aeross  the  line  until  reaehing  the  eolumn  that  shows  the  difference  between  the  highest 
score  and  the  next  highest  seore. 


Column  One 

The  difference  between  the  two  highest  scores  was: 

Total  number  of 

responses 

is: 

Zero. 

They 

were 

equal. 

1 

2 

3 

4 

5 

6  or 

more 

less  than  17 

Multi¬ 

modal 

Multi¬ 

modal 

Mild 

Strong 

Very 

Strong 

Very 

Strong 

Very 

Strong 

between  17  and 

22 

Multi¬ 

modal 

Multi¬ 

modal 

Multi¬ 

modal 

Mild 

Strong 

Very 

Strong 

Very 

Strong 

between  23  and 

26 

Multi¬ 

modal 

Multi¬ 

modal 

Multi¬ 

modal 

Multi¬ 

modal 

Mild 

Strong 

Very 

Strong 

more  than  26 

Multi¬ 

modal 

Multi¬ 

modal 

Multi¬ 

modal 

Multi¬ 

modal 

Multi¬ 

modal 

Mild 

Strong 

Result  Table: 


The  SI 

trength  of  a  single  preference  is: 

Mild 

Strong 

Very  strong 
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Appendix  J. 

Study  Experience  Report. 


Participant  # 


How  many  hours  per  day  (average)  during  the  past  week  do  you  estimate  that  you  spent  studying 
the  emergency  procedures  presented  as  part  of  this  research? 
_ hours. 

IPS  group  only:  How  many  hours  per  day  (average)  during  the  past  week  do  you  estimate  that 
you  spent  using  the  symbols  to  study  the  emergency  procedures  presented  as  part  of  this 
research?  (Note  that  both  totals  can  equal  the  same  number  of  hours.) 

hours. 
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Appendix  K. 

General  information  briefing. 


Slide  1 


General  Information  About  the 
Research  Project 


Arthur  Estrada 
Principal  Investigator 

US  Army  Aeromedical  Research  Laboratory 
Fort  Rucker,  AL 
255-6928 


Slide  2 


The  Research 

•  This  research  is  being  conducted  to  determine  the 
merits  of  two  training  methods  for  memorizing 
aviation  emergency  procedures. 

•  In  order  to  evaluate  the  two  methods,  some  of  you 
will  be  taught  using  Method  1,  while  some  of  you 
will  be  taught  using  Method  2.  I  will  then  compare 
the  performance  of  each  group  (through  written 
tests)  to  see  which  group  gets  the  highest  grades. 
We  will  also  be  asking  your  opinions  of  the 
methods  upon  completion  of  the  training. 
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Slide  3 


Your  Task  in  this  Research 


•  You  will  be  provided  instruction  on  17  UH60  Black  Hawk 
emergency  procedures.  Your  task  is  to  memorize  each 
of  the  selected  emergency  procedure  steps,  regardless 
of  whether  the  step  is  underlined  or  non-underlined. 

•  The  steps  requiring  memorization  are  the  numbered 
steps.  Non-numbered  steps  need  not  be  memorized 
unless  there  is  only  one  step  in  the  procedure. 

•  In  order  to  participate,  you  must  be  willing  to  return  7 
days  after  you  first  receive  instructions  to  take  a  quick 
written  examination  to  check  your  retention.  (You  are 
encouraged  to  study  the  selected  emergency  procedures 
during  the  week  between  instruction  and  testing.) 


Slide  4 


FINALLY: 

•  Remember,  you  have  the  right  to  withdraw  from  the 
study  at  anytime  without  penalty. 

•  Please  DO  NOT  to  discuss  your  assigned  system  of 
memorization  with  others  outside  your  group  as  this  will 
compromise  the  research. 

•  Thank  you  for  agreeing  to  participate. 
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Appendix  L. 

UH-60  aircraft/cockpit  orientation  presentation. 


Slide  1 


UH-60  Aircraft/Cockpit 
Orientation  Presentation 


Slide  2 
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Slide  3 


Slide  4 
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Slide  5 


Slide  6 
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Slide  7 


Slide  8 
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Slide  9 


Slide  10 
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Slide  11 
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Appendix  M. 

Instructional  PowerPoint  presention  (with  IPS  instructions). 


Slide  I 


UH60  Emergency  Procedures 
Class 


Slide  2 


Class  Outline 


1 .  What  is  the  Intuitive  Pictorial  System 
(IPS)? 

2.  What  is  its  purpose? 

3.  Basic  symbology  and  rules. 

4.  Selected  emergency  procedures:  text 
and  symbols. 

5.  Questions. 
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Slide  3 


What  is  the  Intuitive  Pictorial  System  (IPS)? 

The  IPS  is  a  novel  system  of  intuitive  pictures  taught 
as  a  mnemonic  strategy  to  facilitate  learning  and 
prompt  recall  of  complex  emergency  procedures. 

The  pictures  are  characterized  as  intuitive  as  they  are 
formed  with  symbols  representing  aircraft  parts  and 
systems;  and  are  easily  and  immediately 
recognizable  to  pilots.  Thus,  they  should  require  no 
cognitive  effort  in  determining  their  meanings. 


Slide  4 


What  is  its  purpose? 


Based  on  a  survey  by  USAARL,  many  pilots  report  that  memorizing 
and  retaining  emergency  procedures  can  be  difficult  and  requires  a 
considerable  amount  of  time  rehearsing  even  after  years  of  experience 
and  practice.  The  consequences  of  forgetting  them  can  range  from  an 
Unsatisfactory  grade  during  flight  evaluations  to  the  more  serious 
inability  to  respond  to  and  perform  correctly  during  an  actual 
emergency  situation. 

This  method  of  training  may  serve  to  help  to  improve  a  pilot’s  ability  to 
retain  the  emergency  procedures  over  time. 
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Slide  5 


Basic  Symbology  and  Rules 

1 .  Always  use  right  seat  perspective. 

2.  An  arrow  pointing  to  the  right  depicts  “increase”,  an 
arrow  pointing  to  the  left  depicts  “decrease",  and  an 
arrow  pointing  left  AND  right  indicates  “adjust”  (either 
increase  or  decrease,  or  both). 

3.  The  large  symbol  is  the  situation,  the  smaller  symbols 
are  the  procedural  steps. 

4.  “E”  refers  to  engine,  “T”  refers  to  main  rotor,  and 
refers  to  shutdown. 

5.  “X”  over  the  symbol  depicts  failure,  while  “x”  within  the 
symbol  depicts  chip. 


Slide  6 


Basic  Symbols 

Refer  to  Intuitive  Pictorial  System  Symbols 
and  Rules  study  handout.* 


*  Available  in  Appendix  B 
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EMERGENCY  PROCEDURES 


WITH 

SYMBOLS 


EMERGENCY  ENG  SHUTDOWN 


1.  ENG  POWER  CONT  leverfs^  -  OFF. 

2.  ENG  FUEL  SYS  selectorfs^  -  OFF. 

3.  FUEL  BOOST  PUMP  CONTROL 

switch(es')  -  OFF. 
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Slide  9 


SINGLE-ENGINE  FAILURE 

1 .  Collective  -  Adiust  to  maintain  %  RPM  R. 

2.  External  carao/stores  -  Jettison  (if 
reauiredt. 

If  continued  flight  is  not  possible: 

3.  LAND  AS  SOON  AS  POSSIBLE. 

^  \ 

^  y 

If  continued  flight  is  possible: 

\  T 

4.  Establish  sinale-enaine  airsoeed. 

5.  LAND  AS  SOON  AS  PRACTICABLE. 

T  n 

L\ 

Slide  10 


DUAL-ENGINE  FAILURE 


AUTOROTATE 

T 
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Slide  11 


DECREASING  %  RPM  R 

1 .  Collective  -  Ad  lust  to  control 

%  RPM  R. 

2.  ENG  POWER  CONT  lever  - 

LOCKOUT  low  %  TRQ/TGT 

TEMP  engine.  Maintain  %  TRQ 

T 

approximately  10%  below  other 

engine. 

3.  Land  as  soon  as  practicable. 

V 

Slide  12 


INCREASING  %  RPM  R 

1.  ENG  POWER  CONT  lever  -  Retard  hiah 

%  TRQ/TGT  TEMP  engine,  maintain  % 

TRQ  approximately  10%  below  other 

\ 

engine.  . 

/ 

2.  LAND  AS  SOON  AS  PRACTICABLE.  ^ 

If  effected  engine  does  not  respond  to  ENG  _ _ 

z 

POWER  CONT  lever  movement  in  the  range  I 

between  FLY  and  IDLE,  the  HMU  may  be 
malfunctioning  internally. 

If  this  occurs: 

3.  Establish  single  engine  airspeed. 

4.  Perform  EMERG  ENG  SHUTDOWN 

(affected  enaine). 

5.  Refer  to  single  engine  failure  emergency 

procedure. 
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Slide  13 


ENGINE  COMPRESSOR  STALL 

1.  Collective  -  Reduce. 

If  condition  persists: 

2.  ENG  POWER  CONT  lever  (affected 
engine)  -  Retard  (TGT  TEMP  should 
decrease). 

3.  ENG  POWER  CONT  lever  (affected  engine) 

-FLY. 

If  stall  condition  recurs: 

4.  Establish  single  engine  airspeed. 

5.  EMERG  ENG  SHUTDOWN  (affected 
engine). 

5.  Refer  to  single-engine  failure  emergency 
procedure. 


i  £ 


Slide  14 


ENGINE  CHIP  CAUTION  LIGHT 
APPEARS 

1  ENG  POWER  CONT  lever - 
Retard  to  reduce  torque  on 
affected  engine. 

If  oil  pressure  is  below  minimum 
limits  or  if  oil  temperature 
remains  above  maximum  limits: 

2.  EMER  ENG  SHUTDOWN 
(affected  engine). 

3.  Refer  to  single-engine  failure 
emergency  procedure. 
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Slide  15 


ENGINE  HIGH-SPEED  SHAFT 
FAILURE 

1.  Collective  -  Adjust. 

2.  Establish  single  engine  airspeed. 

3.  EMERG  ENG  SHUTDOWN 
(affected  engine).  Do  not  attempt 
to  restart. 

4.  Refer  to  single-engine  failure 
emergency  procedure. 


Slide  16 


LIGHTNING 

STRIKE 

1 .  ENG  POWER  CONT  levers  -  Adjust 
as  required  to  control  %  RPM. 

2.  LAND  AS  SOON  AS  POSSIBLE. 

T 

tt\ 
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Slide  17 


LOSS  OF  TAIL  ROTOR  THRUST 


1.  AUTOROTATE. 

2.  ENG  POWER  CONT  levers  -  OFF 
(when  intended  point  of  landing  is 
assured). 


Slide  18 


LOSS  OF  TAIL  ROTOR  THRUST 
at  LOW  AIRSPEED/HOVER 


1.  Collective  -  Reduce. 

2.  ENG  POWER  CONT  levers  -  OFF 
(5  to  10  feet  above  touchdown). 
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Slide  19 


Slide  20 


CHIP  INPUT  MODULE  LH  or  RH 


2.  LAND  AS  SOON  AS  POSSIBLE. 


Slide  21 


Slide  22 


QUESTIONS? 


Slide  3 


EMERGENCY  ENG  SHUTDOWN 


1.  ENG  POWER  CONT  leverfs^  -  OFF. 

2.  ENG  FUEL  SYS  selectorfsl  -  OFF. 

3.  FUEL  BOOST  PUMP  CONTROL 

switchfes^  -  OFF. 


Slide  4 


SINGLE-ENGINE  FAILURE 

1.  Collective  -  Adjust  to  maintain  %  RPIM  R. 

2.  External  carao/stores  -  Jettison  (if 
required). 

If  continued  flight  is  not  possible: 

3.  LAND  AS  SOON  AS  POSSIBLE. 

If  continued  flight  is  possible: 

4.  Establish  sinale-enaine  airspeed. 

5.  LAND  AS  SOON  AS  PRACTICABLE. 
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Slide  7 


INCREASING  %  RPM  R 

1.  ENG  POWER  CONT  lever  -  Retard  high 

%  TRQ/TGT  TEMP  engine,  maintain  % 

TRQ  approximateiy  10%  below  other 
engine. 

2.  LAND  AS  SOON  AS  PRACTICABLE, 

If  effected  engine  does  not  respond  to  ENG 
POWER  CONT  lever  movement  in  the  range 
between  FLY  and  IDLE,  the  HMD  may  be 
malfunctioning  internally. 

If  this  occurs: 

3.  Establish  single  engine  airspeed. 

4.  Perform  EMERG  ENG  SHUTDOWN 
(affected  engine). 

5.  Refer  to  single  engine  failure  emergency 
procedure. 


Slide  8 


ENGINE  COMPRESSOR  STALL 

1.  Collective  -  Reduce. 

If  condition  persists: 

2.  ENG  POWER  CONT  lever  (affected 
enqinet  -  Retard  (TGT  TEMP  should 
decrease). 

3.  ENG  POWER  CONT  lever  (affected  engine) 

-FLY. 

If  stall  condition  recurs: 

4.  Establish  sinale  enaine  airspeed. 

5.  EMERG  ENG  SHUTDOWN  (affected 
engine). 

5.  Refer  to  single-engine  failure  emergency 
procedure. 
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Slide  9 


ENGINE  CHIP  CAUTION  LIGHT 
APPEARS 

1  ENG  POWER  CONT  lever - 
Retard  to  reduce  torque  on 
affected  engine. 

If  oil  pressure  is  below  minimum 
limits  or  if  oil  temperature 
remains  above  maximum  limits: 

2.  EMER  ENG  SHUTDOWN 
(affected  engine). 

3.  Refer  to  single-engine  failure 
emergency  procedure. 


Slide  10 


ENGINE 

HIGH-SPEED  SHAFT 

FAILURE 

1. 

Collective  -  Adjust. 

2. 

Establish  sinale  enaine  airspeed. 

3. 

EMERG  ENG  SHUTDOWN 
(affected  engine).  Do  not  attempt 
to  restart. 

4. 

Refer  to  single-engine  failure 
emergency  procedure. 
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Slide  11 


LIGHTNING  STRIKE 

1.  ENG  POWER  CONT  levers  -  Adjust 
as  required  to  control  %  RPM. 

2.  LAND  AS  SOON  AS  POSSIBLE. 


Slide  12 


LOSS  OF  TAIL  ROTOR  THRUST 


1.  AUTOROTATE. 

2.  ENG  POWER  CONT  levers  -  OFF 
(when  intended  point  of  landing  is 
assured). 
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Slide  13 


Slide  14 


Slide  15 
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CHIP  INPUT  MODULE  LH  or  RH 


1,  ENG  POWER  CONT  lever  on 
affected  engine  -  IDLE. 

2,  LAND  AS  SOON  AS  POSSIBLE. 


Slide  16 


CHIP  MAIN  MODULE  SUMP,  CHIP 
ACCESSORY  MODULE  LH  or  RH 


LAND  AS  SOON  AS  POSSIBLE. 
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Appendix  O. 
Supplemental  results. 


MG1  Tradtional  MG2Traditional  MGSTradtional 


Membership  Group 


Figure  0-1.  Day  1  EP  Knowledge  Pretest  means. 


MG1  Traditional  MG2Tradtional  MGSTraditional 


Membership  Group 


Figure  0-2.  Day  1  EP  Knowledge  Posttest  means. 
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Membership  Group 


Figure  0-3.  Day  7  EP  Knowledge  Posttest  means. 
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Learning  Style 
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■  Multimodal 

□  Read/Write 

□  Kinesthetic 


Subjective  Assessment 
Category 


Figure  0-4.  MG  1  Learning  Style  Preferenee  by  IPS  Subjeetive  Assessment. 
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Learning  Style 
Preference 
■  Multimodal 

□  Read/Write 

□  Kinesthetic 


Figure  0-5.  MG  2  Learning  Style  Preferenee  by  IPS  Subjeetive  Assessment. 
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Figure  0-6.  MG  3  Learning  Style  Preference  by  IPS  Subjective  Assessment. 
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Figure  0-7.  MG  Aggregate  Learning  Style  Preferenee  by  IPS  Subjeetive  Assessment. 
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Appendix  P. 

Subjective  Assessment  Survey  comments. 


1 .  The  symbols  were  very  intuitive  and  would  be  much  easier  to  identify  in  an  emergency 
situation  than  the  traditional  numbered  steps.  Reading  takes  more  time  than  looking  at  a  single 
symbol  which  represents  multiple  words. 

2.  For  engine  power  control  lever  to  resemble  actual:  [Subject  drew  in  alternative  symbol.] 

3.  I  think  the  system  has  really  great  potential.  I  think  the  pictures  definitely  allowed  me  the 
ability  to  memorize  the  verbiage  a  lot  easier  than  just  memorizing  the  words.  I  didn’t  study  as 
much  as  I  would  have  liked  to  and  I  think  that  if  I  would  I  would  have  been  successful  with  the 
tests.  I  look  forward  to  using  this  type  of  studying  to  help  me  later  on  in  flight  school.  Thanks 
for  developing  and  giving  us  this  resource.  Sorry,  one  more  thought. .  .1  was  wondering  if  there 
is  a  way  to  incorporate  all  steps  of  the  EP  into  one  picture  so  all  you  have  to  look  at  is  one  image. 
Just  a  thought.  Thanks  again! 

4.  I  think  IPS  is  great.  I  would  adjust  location  of  symbols.  Engine  compressor  stall  looks 
complex  at  first.  The  way  I  would  do:  [Subject  drew  alternative  symbols  above  EP  steps  for 
compressor  stalk]  I  think  the  IPS  would  cut  down  on  reaction  time  in  dealing  with  an 
emergency. 

5.  [The  symbols  for]  Quadrant  Eailure  /  Eoss  to  tail  rotor  thrust  may  be  hard  to  find  quickly 
if  using  symbols  in  cockpit  might  get  blurred.  (But  I  have  never  been  in  one  yet,  so  unsure.) 

6.  I  really  liked  the  system.  With  all  systems  (learning)  the  most  benefit  is  gleaned  when  it 
is  adapted  to  the  individual.  In  my  studies  I  incorporated  flash  cards  and  leaned  heavily  on  my 
aircraft  mechanic  to  memorize  the  material.  I  felt  when  I  adapted  the  symbols  with  my  other 
study  methods,  my  retention  quickened  greatly. 

7.  I  have  used  and  memorized  emergency  checklists  in  the  past  and  I  often  have  to 
memorize  the  checklist  by  taking  mental  pictures  of  the  way  the  text  appeared  in  the  checklist.  I 
believe  that  the  IPS  made  memorizing  these  procedures  very  easy  and  less  confusing  than  trying 
to  memorize  text.  In  fact  I  didn’t  even  need  to  study  the  text  to  learn  the  procedures  after  I 
quickly  learned  what  the  pictures  meant.  I  would  not  change  anything  and  I  not  only  recommend 
but  hope  that  this  or  a  similar  method  of  checklist  memorization  is  implemented.  I  believe  that 
[the]  IPS  would  free  up  my  mental  energy  in  an  emergency  situation  as  opposed  to  having  to  rely 
on  text. 

8.  Personally,  I  found  that  the  symbols  made  it  easier  to  recall  the  proper  procedure  to 
perform  in  my  mind  when  looking  at  it  on  paper  vs.  reading  text. 

9.  I  think  for  the  fuel  and  engine  control  diagrams  a  letter  could  be  used  to  annotate  position 
instead  of  positioning  the  diagram  differently  [Subject  drew  in  alternative  symbols].  Overall,  for 
myself  the  picture  reference  helped  me  a  lot.  I  only  studied  once  or  twice  for  about  an  hour  [and 
one-half]  only  paying  half  attention,  but  it  stuck  fairly  well. 

10.  I  had  trouble  at  first  distinguishing  between  engine  fuel  and  fuel  boost  pump.  [T]hose  are 
the  only  symbols  I  had  to  learn.  The  fuel  off  looks  backwards  to  the  other  fuel  positions. 

11.  I  don’t  feel  that  IPS  helped  recall  the  procedures  in  order.  It  did  help  significantly  as  far 
as  quickly  identifying  the  necessary  procedures.  Possibly  putting  the  procedures  (icons)  from 
left  to  right  might  make  remembering  the  order  a  little  easier. 

12.  The  system  really  helped  me.  I  only  wish  I  made  more  time  for  myself  to  study.  If  I  had 
made  more  time  for  myself,  this  system  would  be  great,  easy  to  learn  from. 


124 


13.  As  an  instructor,  it  is  evident  some  students  have  difficulty  memorizing  text  proeedures. 
Using  visual  symbols/aids  for  memorizing  theses  proeedures  1  think  would  greatly  benefit  visual 
learners.  To  limit  ourselves  to  one  style  of  teaehing  restriets  our  effeetiveness  and  ultimately 
reduees  the  effeetiveness  of  erewmembers  who  eould  benefit  from  this  teehnique.  After  the 
short  elass  on  the  visual  procedures,  their  meanings  and  the  EP’s  themselves,  the  dual  engine 
failure  EP  eame  up  and  1  immediately  saw  [it]  in  my  mind  [Subjeet  drew  in  the  IPS  symbol.] 

14.  Good  system  for  new  aviators.  A  little  diffieult  for  those  [who]  have  already  learned  the 
EP. 

15.  The  IPS  did  not  assist  me  beeause  I  already  had  them  memorized.  It  did  help  me  reeall 
speeifie  proeedures  (meaning  assoeiating  a  pieture  for  the  EP).  [T]he  individual  steps  helped  me 
organize  what  I  already  knew  better;  i.e.,  it  didn’t  help  me  to  assoeiate  pietures  with  every  step  in 
an  EP,  but  it  did  help  me  reeall  the  steps  from  memory  faster  when  I  saw  the  pieture  assoeiated 
with  the  EP. 

16.  In  terms  of  questions  13  and  14,  using  the  IPS  eheeklist  alone  during  an  actual  EP,  I  feel, 
may  induee  more  of  a  hazard.  What  should  happen  if,  in  the  event  of  an  EP  during  the  eonfusion 
in  the  eockpit  at  the  time  a  pilot  would  misinterpret  a  symbol;  he  or  she  might  induee  more  of  a 
hazard  than  the  actual  EP  itself  Now,  used  in  eonjunction  with  the  worded  text  or  used  as  a 
study  tool,  I  believe,  would  be  a  great  sueeess  (i.e.,  the  missed  approaeh  proeedures  in  the 
approaeh  plates  have  pietures  as  well  as  words.)  In  the  heat  of  the  moment,  pietures  work  better, 
but  the  words  aceompanying  work  well,  too! 

17.  I  studied  for  a  grand  total  of  15  min  and  only  got  a  little  eonfused  with  PCE  and  fuel 
system  seleetors. 

18.  Some  of  the  symbols  should  be  refined  to  be  more  easily  identified  by  aviators.  The 
aetual  symbol  set  for  the  EP  should  be  more  structured  before  being  implemented. 

19.  If  I  would  have  spent  more  time  studying  this  eould  be  worthwhile.  I  would  like  to  see 
an  EP  in  eheeklist  form  to  further  my  opinion. 

20.  It  took  me  awhile  to  figure  the  PCE  and  Euel  system  seleetor.  [I]  did  not  assoeiate  the 
symbol  to  the  item  at  first. 

21 .  I  don’t  believe  that  the  initial  intuitiveness  of  the  symbols  has  mueh  bearing  on  the  value 
of  this  eoneept.  Onee  learned  the  symbols  beeome  intuitive  (i.e.,  eheeklist  symbols).  I  do 
believe  that  this  is  a  mueh  more  simple  [and]  easy  way  to  learn  [and]  recall  the  emergency 
procedures.  I  have  to  think  of  all  the  words  in  an  EP  [and]  that  takes  time.  To  think  of  or  see  a 
eouple  symbols  as  a  referenee  is  mueh  easier  and  eonveys  more  with  less.  It  also  aids  in 
eorrelation  of  notions  to  events.  I  would  like  to  see  this  (or  something  similar)  adopted  into 
training.  It  seems  like  it  would  have  value  in  the  air;  [flying]  with  “glass  oockpits”  as  well. 

22.  With  many  hours  and  years  of  experienoe,  I  found  IPS  to  be  an  exoellent  addition  to  my 
leaming/retention.  I  strongly  support  oontinued  study  and  integration  into  offioial  flight  training 
syllabus.  We  must  oontinue  forward  with  this  promising  program. 

23.  Great  idea,  however  the  symbols  eould  be  improved  by  using  more  detailed  pietures 
instead  of  the  [subject  drew  in  stiek  figures]. 

24.  This  system  eould  be  useful  if  started  early  in  the  flight  sehool  program  and  reinforeed 
throughout.  Perhaps  some  type  of  informed  daily  questions  eoneeming  the  symbols  as  a  part  of 
student  flight  briefing  would  offer  good  reinforcement. 

25.  Interesting  eoneept.  The  symbols  used  probably  need  to  be  revised. 

26.  Some  of  the  symbols,  i.e.,  external  stores  and  eompressor  stall  are  diffieult  to 
eoneeptualize  and  eommit  to  memory. 
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27.  Use  the  color  red  for  all  the  [land  as  soon  as  possible  and]  autorotate. 

28.  Seems  to  be  a  much  easier  system  to  learn  than  having  to  learn  the  whole  sentence. 

29.  [The]  symbol  for  fuel  system  selector  (the  bend)  should  stay  consistent  for  all  three 
positions.  On  a  malfunction  display,  symbols  merely  add  a  step  which  requires  more 
interpretation  and  therefore,  more  time  to  process  and  a  greater  risk  of  error.  If  an  MFD  is 
capable  of  assisting  in  an  emergency,  why  not  simply  display  the  text?  We're  only  taling  about 
underlined  steps  here. 

30.  Some  symbols  weren’t  intuitive  (as  demonstrated  by  symbols  posttest),  but  overall  a 
useful  system. 

3 1 .  Establish  single-engine  airspeed  symbol  is  not  intuitive. 

32.  Fuel  system  selector  position  was  not  intuitive  (cross  feed,  direct,  off).  High  speed  shaft 
failure  [was]  not  intuitive.  [The]  IPS  system  seems  like  it  would  be  an  additional  step  to 
memorize.  Aviators  would  still  need  to  memorize  written  steps,  and  the  IPS  would  add 
confusion. 
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